Обзор методов глубокого обучения для суммаризации кода

Глубокие нейронные модели стали передовым подходом в задаче суммаризации кода. До эпохи больших языковых моделей многие специализированные модели применяли различные методы для улучшения базовых моделей преобразования последовательностей, используя структуру кода, за счёт сходства фрагментов кода и их описаний, а также применяя многозадачное обучение. В данном обзоре выделяются ключевые архитектурные компоненты этих методов и предлагается их таксономия. Также описывается, как разные модели используют один и тот же метод или как одна модель сочетает несколько методов. Кроме того, мы обсуждаем текущие недостатки этих моделей, потенциальные направления дальнейших исследований и применимость больших языковых моделей в задаче суммаризации кода, сравнивая их с рассмотренными подходами.

1. Ahmad W.U., Chakraborty S., Ray B., Chang K.-W. A Transformer-based Approach for Source Code Summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 4998-5007. Association for Computational Linguistics, 2020. DOI: 10.18653/v1/2020.acl-main.449.

2. Iyer S., Konstas I., Cheung A., Zettlemoyer L. Summarizing Source Code using a Neural Attention Model. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, August 7-12, 2016, Volume 1: Long Papers. The Association for Computer Linguistics, 2016. DOI: 10.18653/v1/p16-1195.

3. Jiang S., Armaly A., McMillan C. Automatically Generating Commit Messages from Diffs using Neural Machine Translation. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, ASE 2017, Urbana, IL, USA, October 30 - November 03, 2017, pp. 135-146. IEEE Computer Society, 2017. DOI: 10.1109/ASE.2017.8115626.

4. Loyola P., Marrese-Taylor E., Matsuo Y. A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, pp. 287-292. Association for Computational Linguistics, 2017. DOI: 10.18653/v1/P17-2045.

5. Liu Z., Xia X., Hassan A.E., Lo D., Xing Z., Wang X. Neural-Machine-Translation-Based Commit Message Generation: How Far Are We? In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, Montpellier, France, September 3-7, 2018, pp. 373-384. ACM, 2018. DOI: 10.1145/3238147.3238190.

6. Li M., Yu H., Fan G., Zhou Z., Huang J. ClassSum: a deep learning model for class-level code summarization. Neural Computing and Applications, 2023, vol. 35, no. 4, pp. 3373-3393. DOI: 10.1007/s00521-022-07877-z.

7. Malhotra M., Chhabra J. Class Level Code Summarization Based on Dependencies and Micro Patterns. 2018, pp. 1011-1016. DOI: 10.1109/ICICCT.2018.8473199.

8. Makharev V., Ivanov V. Code Summarization Beyond Function Level. 2025. DOI: 10.48550/arXiv.2502.16704.

9. Ma Y., Yang Q., Cao R., Li B., Huang F., Li Y. Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration. 2025. DOI: 10.48550/arXiv.2406.01422.

10. Lomshakov V., Podivilov A., Savin S., Baryshnikov O., Lisevych A., Nikolenko S. ProConSuL: Project Context for Code Summarization with LLMs. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, Miami, Florida, US, November 2024, pp. 866-880. Association for Computational Linguistics, 2024. DOI: 10.18653/v1/2024.emnlp-industry.65.

11. GitHub. Available at: https://github.com/. Accessed 05.06.2026.

12. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin I. Attention Is All You Need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA, December 4-9, 2017, pp. 5998-6008, 2017.

13. Hochreiter S., Schmidhuber J. Long Short-Term Memory. Neural Computation, 1997, vol. 9, no. 8, pp. 1735-1780.

14. Cho K., van Merrienboer B., Gulcehre C., Bahdanau D., Bougares F., Schwenk H., Bengio Y. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Doha, Qatar, October 25-29, 2014, pp. 1724-1734. ACL, 2014.

15. Schuster M., Paliwal K.K. Bidirectional Recurrent Neural Networks. IEEE Transactions on Signal Processing, 1997, vol. 45, no. 11, pp. 2673-2681.

16. Luong T., Pham H., Manning C.D. Effective Approaches to Attention-based Neural Machine Translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pp. 1412-1421. The Association for Computational Linguistics, 2015. DOI: 10.18653/V1/D15-1166.

17. Hu X., Li G., Xia X., Lo D., Jin Z. Deep code comment generation. In: 2018 IEEE/ACM 26th International Conference on Program Comprehension, ICPC 2018, pp. 200-210. IEEE, 2018.

18. Shi E., Wang Y., Du L., Chen J., Han S., Zhang H., Zhang D., Sun H. On the Evaluation of Neural Code Summarization. In: 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022, pp. 1597-1608. ACM, 2022. DOI: 10.1145/3510003.3510060.

19. Barone A.V.M., Sennrich R. A parallel corpus of python functions and documentation strings for automated code documentation and code generation. CoRR, 2017, abs/1707.02275. DOI: 10.48550/arXiv.1707.02275.

20. Wan Y., Zhao Z., Yang M., Xu G., Ying H., Wu J., Yu P.S. Improving Automatic Source Code Summarization via Deep Reinforcement Learning. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, Montpellier, France, September 3-7, 2018, pp. 397-407. ACM, 2018. DOI: 10.1145/3238147.3238206.

21. Hu X., Li G., Xia X., Lo D., Lu S., Jin Z. Summarizing Source Code with Transferred API Knowledge. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, Stockholm, Sweden, July 13-19, 2018, pp. 2269-2275. ijcai.org, 2018. DOI: 10.24963/ijcai.2018/314.

22. LeClair A., Jiang S., McMillan C. A neural model for generating natural language summaries of program subroutines. In: Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019, pp. 795-806. IEEE/ACM, 2019. DOI: 10.1109/ICSE.2019.00087.

23. UCI Source Code Data Sets. Available at: https://ics.uci.edu/~lopes/datasets/. Accessed 05.06.2026.

24. Husain H., Wu H.-H., Gazit T., Allamanis M., Brockschmidt M. CodeSearchNet Challenge: Evaluating the State of Semantic Code Search. CoRR, 2019, abs/1909.09436. DOI: 10.48550/arXiv.1909.09436.

25. Hu X., Li G., Xia X., Lo D., Jin Z. Deep Code Comment Generation with Hybrid Lexical and Syntactical Information. Empirical Software Engineering, 2020, vol. 25, no. 3, pp. 2179-2217. DOI: 10.1007/s10664-019-09730-9.

26. Liu S., Chen Y., Xie X., Siow J.K., Liu Y. Retrieval-Augmented Generation for Code Summarization via Hybrid GNN. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.

27. Shi L., Mu F., Chen X., Wang S., Wang J., Yang Y., Li G., Xia X., Wang Q. Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization. In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022, Singapore, Singapore, pp. 107-119. Association for Computing Machinery, 2022. DOI: 10.1145/3540250.3549145.

28. Papineni K., Roukos S., Ward T., Zhu W.J. BLEU: a Method for Automatic Evaluation of Machine Translation. 2002. DOI: 10.3115/1073083.1073135.

29. NLTK Source. Available at: https://github.com/nltk/nltk. Accessed 05.05.2026.

30. Lin C.-Y. ROUGE: A Package for Automatic Evaluation of Summaries. In: Text Summarization Branches Out, Barcelona, Spain, July 2004, pp. 74-81. Association for Computational Linguistics, 2004

31. Banerjee S., Lavie A. METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005, Ann Arbor, Michigan, USA, June 29, 2005, pp. 65-72. Association for Computational Linguistics, 2005.

32. Vedantam R., Zitnick C.L., Parikh D. CIDEr: Consensus-Based Image Description Evaluation. CoRR, 2014, abs/1411.5726. DOI: 10.48550/arXiv.1411.5726.

33. Zhang T., Kishore V., Wu F., Weinberger K.Q., Artzi Y. BERTScore: Evaluating Text Generation with BERT. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.

34. Allamanis M., Barr E.T., Devanbu P., Sutton C. A survey of machine learning for big code and naturalness. ACM Computing Surveys (CSUR), 2018, vol. 51, no. 4, article 81.

35. Niu C., Li C., Ng V., Ge J., Huang L., Luo B. SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations. In: 44th IEEE/ACM International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022, pp. 1-13. ACM, 2022. DOI: 10.1145/3510003.3510096.

36. Yang G., Chen X., Cao J., Xu S., Cui Z., Yu C., Liu K. ComFormer: Code Comment Generation via Transformer and Fusion Method-Based Hybrid Code Representation. In: 8th International Conference on Dependable Systems and Their Applications, DSA 2021, Yinchuan, China, August 5-6, 2021, pp. 30-41. IEEE, 2021. DOI: 10.1109/DSA52907.2021.00013.

37. Choi Y., Bak J., Na C., Lee J.-H. Learning Sequential and Structural Information for Source Code Summarization. In: Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, pp. 2842-2851. Association for Computational Linguistics, 2021. DOI: 10.18653/v1/2021.findings-acl.251.

38. Tang Z., Shen X., Li C., Ge J., Huang L., Zhu Z., Luo B. AST-Trans: Code Summarization with Efficient Tree-Structured Attention. In: 44th IEEE/ACM International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022, pp. 150-162. ACM, 2022. DOI: 10.1145/3510003.3510224.

39. Zhang J., Wang X., Zhang H., Sun H., Liu X. Retrieval-Based Neural Source Code Summarization. In: ICSE '20, Seoul, South Korea, pp. 1385-1397. New York, NY, USA, Association for Computing Machinery, 2020. DOI: 10.1145/3377811.3380383.

40. Choi Y., Na C., Kim H., Lee J.-H. READSUM: Retrieval-Augmented Adaptive Transformer for Source Code Summarization. IEEE Access, 2023, vol. 11, pp. 51155-51165.

41. Villmow J., Ulges A., Schwanecke U. A Structural Transformer with Relative Positions in Trees for Code-to-Sequence Tasks. In: International Joint Conference on Neural Networks, IJCNN 2021, Shenzhen, China, July 18-22, 2021, pp. 1-10. IEEE, 2021.

42. Shi Y., Yin Y., Wang Z., Lo D., Zhang T., Xia X., Zhao Y., Xu B. How to Better Utilize Code Graphs in Semantic Code Search? In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022, Singapore, Singapore, pp. 722-733. Association for Computing Machinery, 2022. DOI: 10.1145/3540250.3549087.

43. Wang W., Zhang Y., Sui Y., Wan Y., Zhao Z., Wu J., Yu P., Xu G. Reinforcement-learning-guided Source Code Summarization via Hierarchical Attention. IEEE Transactions on Software Engineering, 2020.

44. Son J., Hahn J., Seo H., Han Y.-S. Boosting Code Summarization by Embedding Code Structures. In: Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pp. 5966-5977. International Committee on Computational Linguistics, 2022.

45. Gao Y., Lyu C. M2TS: multi-scale multi-modal approach based on transformer for source code summarization. In: Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension, ICPC 2022, Virtual Event, May 16-17, 2022, pp. 24-35. ACM, 2022.

46. Guo J., Liu J., Wan Y., Li L., Zhou P. Modeling Hierarchical Syntax Structure with Triplet Position for Source Code Summarization. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 486-500. Association for Computational Linguistics, 2022. DOI: 10.18653/v1/2022.acl-long.37.

47. LeClair A., Haque S., Wu L., McMillan C. Improved code summarization via a graph neural network. In: Proceedings of the 28th International Conference on Program Comprehension, pp. 184-195, 2020.

48. Lin C., Ouyang Z., Zhuang J., Chen J., Li H., Wu R. Improving Code Summarization with Block-Wise Abstract Syntax Tree Splitting. In: 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021, pp. 184-195. IEEE, 2021.

49. Shido Y., Kobayashi Y., Yamamoto A., Miyamoto A., Matsumura T. Automatic Source Code Summarization with Extended Tree-LSTM. In: International Joint Conference on Neural Networks, IJCNN 2019, Budapest, Hungary, July 14-19, 2019, pp. 1-8. IEEE, 2019. DOI: 10.1109/IJCNN.2019.8851751.

50. Zhang C., Zhou Q., Qiao M., Tang K., Xu L., Liu F. Re_Trans: Combined Retrieval and Transformer Model for Source Code Summarization. Entropy, 2022, vol. 24, no. 10, article 1372.

51. Gao S., Gao C., Zeng J., Nie L.Y., Xia X., Lyu M. Code Structure Guided Transformer for Source Code Summarization. ACM Transactions on Software Engineering and Methodology, 2022. DOI: 10.1145/3522674.

52. Gong Z., Gao C., Wang Y., Gu W., Peng Y., Xu Z. Source Code Summarization with Structural Relative Position Guided Transformer. In: IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022, Honolulu, HI, USA, March 15-18, 2022, pp. 13-24. IEEE, 2022. DOI: 10.1109/SANER53432.2022.00013.

53. Wu H., Zhao H., Zhang M. Code Summarization with Structure-induced Transformer. In: Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, pp. 1078-1090. Association for Computational Linguistics, 2021. DOI: 10.18653/v1/2021.findings-acl.93.

54. Zügner D., Kirschstein T., Catasta M., Leskovec J., Günnemann S. Language-Agnostic Representation Learning of Source Code from Structure and Context. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.

55. Guo D., Ren S., Lu S., Feng Z., Tang D., Liu S., Zhou L., Duan N., Svyatkovskiy A., Fu S., Tufano M., Deng S.K., Clement C.B., Drain D., Sundaresan N., Yin J., Jiang D., Zhou M. GraphCodeBERT: Pre-training Code Representations with Data Flow. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.

56. Jiang X., Zheng Z., Lyu C., Li L., Lyu L. TreeBERT: A Tree-Based Pre-Trained Model for Programming Language. In: Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, UAI 2021, Virtual Event, July 27-30, 2021, vol. 161 of Proceedings of Machine Learning Research, pp. 54-63. AUAI Press, 2021.

57. Alon U., Brody S., Levy O., Yahav E. code2seq: Generating Sequences from Structured Representations of Code. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.

58. Wei B., Li G., Xia X., Fu Z., Jin Z. Code Generation as a Dual Task of Code Summarization. In: Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Vancouver, BC, Canada, December 8-14, 2019, pp. 6559-6569, 2019.

59. Xie R., Ye W., Sun J., Zhang S. Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-task Learning Approach. In: 29th IEEE/ACM International Conference on Program Comprehension, ICPC 2021, Madrid, Spain, May 20-21, 2021, pp. 138-148. IEEE, 2021. DOI: 10.1109/ICPC52881.2021.00022.

60. Li J., Li Y., Li G., Hu X., Xia X., Jin Z. EditSum: A Retrieve-and-Edit Framework for Source Code Summarization. In: 36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021, Melbourne, Australia, November 15-19, 2021, pp. 155-166. IEEE, 2021. DOI: 10.1109/ASE51524.2021.9678724.

61. Parvez M.R., Ahmad W.U., Chakraborty S., Ray B., Chang K.-W. Retrieval Augmented Code Generation and Summarization. In: Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, November 16-20, 2021, pp. 2719-2734. Association for Computational Linguistics, 2021. DOI: 10.18653/v1/2021.findings-emnlp.232.

62. Yao Z., Peddamail J.R., Sun H. CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning. CoRR, 2019, abs/1904.00720. DOI: 10.48550/arXiv.1904.00720.

63. Ye T., Wu L., Ma T., Zhang X., Du Y., Liu P., Wang W., Ji S. Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization. CoRR, 2023, abs/2305.11074. DOI: 10.48550/arXiv.2305.11074.

64. Guo D., Lu S., Duan N., Wang Y., Zhou M., Yin J. UniXcoder: Unified Cross-Modal Pre-Training for Code Representation. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 7212-7225. Association for Computational Linguistics, 2022.

65. Feng Z., Guo D., Tang D., Duan N., Feng X., Gong M., Shou L., Qin B., Liu T., Jiang D., Zhou M. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In: Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, November 16-20, 2020, pp. 1536-1547. Association for Computational Linguistics, 2020. DOI: 10.18653/v1/2020.findings-emnlp.139.

66. Kanade A., Maniatis P., Balakrishnan G., Shi K. Learning and Evaluating Contextual Embedding of Source Code. In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, Virtual Event, July 13-18, 2020, vol. 119 of Proceedings of Machine Learning Research, pp. 5110-5121. PMLR, 2020.

67. Liu F., Li G., Zhao Y., Jin Z. Multi-task Learning Based Pre-trained Language Model for Code Completion. In: 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020, Melbourne, Australia, September 21-25, 2020, pp. 473-485. IEEE, 2020. DOI: 10.1145/3324884.3416591.

68. Wang Y., Wang W., Joty S.R., Hoi S.C.H. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, November 7-11, 2021, pp. 8696-8708. Association for Computational Linguistics, 2021. DOI: 10.18653/v1/2021.emnlp-main.685.

69. Wang Y., Le H., Gotmare A.D., Bui N.D.Q., Li J., Hoi S.C.H. CodeT5+: Open Code Large Language Models for Code Understanding and Generation. CoRR, 2023, abs/2305.07922. DOI: 10.48550/arXiv.2305.07922.

70. Mastropaolo A., Scalabrino S., Cooper N., Nader-Palacio D., Poshyvanyk D., Oliveto R., Bavota G. Studying the Usage of Text-to-Text Transfer Transformer to Support Code-Related Tasks. In: 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, May 22-30, 2021, pp. 336-347. IEEE, 2021. DOI: 10.1109/ICSE43902.2021.00041.

71. Elnaggar A., Ding W., Jones L., Gibbs T., Feher T., Angerer C., Severini S., Matthes F., Rost B. CodeTrans: Towards Cracking the Language of Silicon’s Code through Self-Supervised Deep Learning and High Performance Computing. CoRR, 2021, abs/2104.02443. DOI: 10.48550/arXiv.2104.02443.

72. Haque S., LeClair A., Wu L., McMillan C. Improved Automatic Summarization of Subroutines via Attention to File Context. In: 17th International Conference on Mining Software Repositories, MSR 2020, Seoul, Republic of Korea, June 29-30, 2020, pp. 300-310. ACM, 2020. DOI: 10.1145/3379597.3387449.

73. Zhang J., Panthaplackel S., Nie P., Mooney R.J., Li J.J., Gligoric M. Learning to Generate Code Comments from Class Hierarchies. CoRR, 2021, abs/2103.13426. DOI: 10.48550/arXiv.2103.13426.

74. Ahmad W.U., Chakraborty S., Ray B., Chang K.-W. Unified Pre-training for Program Understanding and Generation. 2021. DOI: 10.48550/arXiv.2103.06333.

75. Du X., Liu M., Wang K., Wang H., Liu J., Chen Y., Feng J., Sha C., Peng X., Lou Y. ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-Level Code Generation. CoRR, 2023, abs/2308.01861. DOI: 10.48550/arXiv.2308.01861.

76. Chen Q., Xia X., Hu H., Lo D., Li S. Why My Code Summarization Model Does Not Work: Code Comment Improvement with Category Prediction. ACM Transactions on Software Engineering and Methodology, 2021, vol. 30, no. 2, pp. 25:1-25:29. DOI: 10.1145/3434280.

77. Andryushchenko, G., Ivanov, V., Makharev, V., Tukhtina, E. and Valeev, A. Leveraging large language models in code question answering: Baselines and issues. In International Conference on Analysis of Images, Social Networks and Texts, pp. 3-17. Springer Nature Switzerland, 2024.

78. GitHub Copilot. Available at: https://github.com/features/copilot. Accessed 10.06.2026.

79. Phind. Available at: https://huggingface.co/Phind. Accessed 05.06.2026.

80. Rozière B., Gehring J., Gloeckle F., Sootla S., Gat I., Tan X.E., Adi Y., Liu J., Remez T., Rapin J., Kozhevnikov A., Evtimov I., Bitton J., Bhatt M., Canton-Ferrer C., Grattafiori A., Xiong W., Défossez A., Copet J., Azhar F., Touvron H., Martin L., Usunier N., Scialom T., Synnaeve G. Code Llama: Open Foundation Models for Code. CoRR, 2023, abs/2308.12950. DOI: 10.48550/arXiv.2308.12950.

81. Ito T., van Deemter K., Suzuki J. Reference-free Evaluation Metrics for Text Generation: A Survey. 2025. DOI: 10.48550/arXiv.2501.12011.

82. Chen M., Tworek J., Jun H., Yuan Q., Pondé de Oliveira Pinto H., Kaplan J., Edwards H., Burda Y., Joseph N., Brockman G., Ray A., Puri R., Krueger G., Petrov M., Khlaaf H., Sastry G., Mishkin P., Chan B., Gray S., Ryder N., Pavlov M., Power A., Kaiser L., Bavarian M., Winter C., Tillet P., Such F.P., Cummings D., Plappert M., Chantzis F., Barnes E., Herbert-Voss A., Guss W.H., Nichol A., Paino A., Tezak N., Tang J., Babuschkin I., Balaji S., Jain S., Saunders W., Hesse C., Carr A.N., Leike J., Achiam J., Misra V., Morikawa E., Radford A., Knight M., Brundage M., Murati M., Mayer K., Welinder P., McGrew B., Amodei D., McCandlish S., Sutskever I., Zaremba W. Evaluating Large Language Models Trained on Code. CoRR, 2021, abs/2107.03374. DOI: 10.48550/arXiv.2107.03374.

83. Li Y., Choi D.H., Chung J., Kushman N., Schrittwieser J., Leblond R., Eccles T., Keeling J., Gimeno F., Dal Lago A., Hubert T., Choy P., de Masson d’Autume C., Babuschkin I., Chen X., Huang P.-S., Welbl J., Gowal S., Cherepanov A., Molloy J., Mankowitz D.J., Sutherland Robson E., Kohli P., de Freitas N., Kavukcuoglu K., Vinyals O. Competition-Level Code Generation with AlphaCode. CoRR, 2022, abs/2203.07814. DOI: 10.48550/arXiv.2203.07814.

84. Li R., Ben Allal L., Zi Y., Muennighoff N., Kocetkov D., Mou C., Marone M., Akiki C., Li J., Chim J., Liu Q., Zheltonozhskii E., Zhuo T.Y., Wang T., Dehaene O., Davaadorj M., Lamy-Poirier J., Monteiro J., Shliazhko O., Gontier N., Meade N., Zebaze A., Yee M.-H., Umapathi L.K., Zhu J., Lipkin B., Oblokulov M., Wang Z., Murthy R.V., Stillerman J.T., Patel S.S., Abulkhanov D., Zocca M., Dey M., Zhang Z., Fahmy N., Bhattacharyya U., Yu W., Singh S., Luccioni S., Villegas P., Kunakov M., Zhdanov F., Romero M., Lee T., Timor N., Ding J., Schlesinger C., Schoelkopf H., Ebert J., Dao T., Mishra M., Gu A., Robinson J., Anderson C.J., Dolan-Gavitt B., Contractor D., Reddy S., Fried D., Bahdanau D., Jernite Y., Muñoz Ferrandis C., Hughes S., Wolf T., Guha A., von Werra L., de Vries H. StarCoder: may the source be with you! Transactions on Machine Learning Research, 2023, vol. 2023.

85. Lozhkov A., Li R., Ben Allal L., Cassano F., Lamy-Poirier J., Tazi N., Tang A., Pykhtar D., Liu J., Wei Y., Liu T., Tian M., Kocetkov D., Zucker A., Belkada Y., Wang Z., Liu Q., Abulkhanov D., Paul I., Li Z., Li W.-D., Risdal M., Li J., Zhu J., Zhuo T.Y., Zheltonozhskii E., Dade N.O.O., Yu W., Krauß L., Jain N., Su Y., He X., Dey M., Abati E., Chai Y., Muennighoff N., Tang X., Oblokulov M., Akiki C., Marone M., Mou C., Mishra M., Gu A., Hui B., Dao T., Zebaze A., Dehaene O., Patry N., Xu C., McAuley J.J., Hu H., Scholak T., Paquet S., Robinson J., Anderson C.J., Chapados N., et al. StarCoder 2 and The Stack v2: The Next Generation. CoRR, 2024, abs/2402.19173. DOI: 10.48550/arXiv.2402.19173.

86. Chowdhery A., Narang S., Devlin J., Bosma M., Mishra G., Roberts A., Barham P., Chung H.W., Sutton C., Gehrmann S., Schuh P., Shi K., Tsvyashchenko S., Maynez J., Rao A., Barnes P., Tay Y., Shazeer N., Prabhakaran V., Reif E., Du N., Hutchinson B., Pope R., Bradbury J., Austin J., Isard M., Gur-Ari G., Yin P., Duke T., Levskaya A., Ghemawat S., Dev S., Michalewski H., Garcia X., Misra V., Robinson K., Fedus L., Zhou D., Ippolito D., Luan D., Lim H., Zoph B., Spiridonov A., Sepassi R., Dohan D., Agrawal S., Omernick M., Dai A.M., Sankaranarayana Pillai T., Pellat M., Lewkowycz A., Moreira E., Child R., Polozov O., Lee K., Zhou Z., Wang X., Saeta B., Diaz M., Firat O., Catasta M., Wei J., Meier-Hellstern K., Eck D., Dean J., Petrov S., Fiedel N. PaLM: Scaling Language Modeling with Pathways. Journal of Machine Learning Research, 2023, vol. 24, pp. 240:1-240:113.

87. Luo Z., Can Xu, Zhao P., Sun Q., Geng X., Hu W., Tao C., Ma J., Lin Q., Jiang D. WizardCoder: Empowering Code Large Language Models with Evol-Instruct. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.

88. Guo D., Zhu Q., Yang D., Xie Z., Dong K., Zhang W., Chen G., Bi X., Wu Y., Li Y.K., Luo F., Xiong Y., Liang W. DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence. CoRR, 2024, abs/2401.14196. DOI: 10.48550/arXiv.2401.14196.

89. Zheng T., Zhang G., Shen T., Liu X., Lin B.Y., Fu J., Chen W., Yue X. OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement. In: Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pp. 12834-12859. Association for Computational Linguistics, 2024.

90. Hui B., Yang J., Cui Z., Yang J., Liu D., Zhang L., Liu T., Zhang J., Yu B., Dang K., Yang A., Men R., Huang F., Ren X., Ren X., Zhou J., Lin J. Qwen2.5-Coder Technical Report. CoRR, 2024, abs/2409.12186. DOI: 10.48550/arXiv.2409.12186.

91. Sun W., Miao Y., Li Y., Zhang H., Fang C., Liu Y., Deng G., Liu Y., Chen Z. Source Code Summarization in the Era of Large Language Models. 2024. DOI: 10.48550/arXiv.2407.07959.

92. Sun W., Fang C., You Y., Miao Y., Liu Y., Li Y., Deng G., Huang S., Chen Y., Zhang Q., Qian H., Liu Y., Chen Z. Automatic Code Summarization via ChatGPT: How Far Are We? CoRR, 2023, abs/2305.12865. DOI: 10.48550/arXiv.2305.12865.

93. Sharma T., Kechagia M., Georgiou S., Tiwari R., Sarro F. A Survey on Machine Learning Techniques for Source Code Analysis. CoRR, 2021, abs/2110.09610. DOI: 10.48550/arXiv.2110.09610.

94. Song X., Sun H., Wang X., Yan J. A Survey of Automatic Generation of Source Code Comments: Algorithms and Techniques. IEEE Access, 2019, vol. 7, pp. 111411-111428.

95. Zhu Y., Pan M. Automatic Code Summarization: A Systematic Literature Review. CoRR, 2019, abs/1909.04352. DOI: 10.48550/arXiv.1909.04352.

96. Yang B., Liping Z., Fengrong Z. A survey on research of code comment. In: Proceedings of the 2019 3rd International Conference on Management Engineering, Software Engineering and Service Sciences, ICMSS 2019, Wuhan, China, January 12-14, 2019, pp. 45-51. ACM, 2019.

97. Zhang C., Wang J., Zhou Q., Xu T., Tang K., Gui H., Liu F. A Survey of Automatic Source Code Summarization. Symmetry, 2022, vol. 14, no. 3, article 471. DOI: 10.3390/sym14030471.