Десятилетие достижений в синтезе программ по спецификациям на естественном языке: систематический обзор литературы

Программный синтез – это процесс автоматического создания программного обеспечения на основе спецификации требований. В этой статье представлен систематический обзор литературы, посвященный синтезу программ из спецификаций, выраженных на естественном языке. Исследуемая проблематика заключается в сложности автоматического создания точного и надежного кода из высокоуровневых, неоднозначных описаний на естественном языке – барьер, который ограничивает более широкое использование средств автоматизации при разработке программного обеспечения. Для исследования этой проблемы авторы систематически изучали работы, опубликованные в период с 2014 по 2024 год, делая акцент на работы, в которых рассматриваются различные подходы к синтезу программ на основе данных на естественном языке. Обзор следует строгой методологии, включающей поисковые строки, адаптированные для сбора соответствующих исследований из пяти основных источников данных: IEEE, ACM, Springer, Elsevier и MDPI. В процессе отбора применялись строгие критерии включения и исключения, что привело к окончательному набору из 20 высококачественных исследований. Результаты показывают значительные достижения в этой области, особенно в интеграции больших языковых моделей (LLM) с методами синтеза программ. Обзор также освещает проблемы и завершается изложением ключевых тенденций и предложением будущих направлений исследований, нацеленных на преодоление этих проблем и расширение применимости синтеза программ в различных областях.

1. J. Fu, F. B. Bastani, and I.-L. Yen, in Innovations for Requirement Analysis. From Stakeholders’ Needs to Formal Designs, edited by B. Paech and C. Martell (Springer Berlin Heidelberg, Berlin, Heidelberg, 2007), pp. 43–61, ISBN 978-3-540-89778-1.

2. Z. Manna and R. J. Waldinger, Toward automatic program synthesis (1971), URL https://doi.org/10.1145/362566. 362568.

3. S. Jiho and N. Jaechang, A survey of automatic code generation from natural language (2021).

4. A. Bandi, P. V. S. R. Adapa, and Y. E. V. P. K. Kuchi, The power of generative ai: A review of requirements, models, input–output formats, evaluation metrics, and challenges (2023), URL https://www.mdpi.com/1999-5903/15/8/260.

5. S. Gulwani, in Formal Methods in Computer Aided Design (2010), pp. 1–1.

6. Z. Manna and R. Waldinger, A deductive approach to program synthesis (1980), URL https://doi.org/10.1145/357084. 357090.

7. E. Kitzelmann, in Approaches and Applications of Inductive Programming, edited by U. Schmid, E. Kitzelmann, and R. Plasmeijer (Springer Berlin Heidelberg, Berlin, Heidelberg, 2010), pp. 50–73, ISBN 978-3-642-11931-6.

8. A. F. Subahi, Cognification of program synthesis—a systematic feature-oriented analysis and future direction (2020), URL https://www.mdpi.com/2073-431X/9/2/27.

9. J. Sauvola, S. Tarkoma, M. Klemettinen, J. Riekki, and D. Doermann, Future of software development with generative ai (2024).

10. C. Ebert and P. Louridas, Generative ai for software practitioners (2023).

11. K. S. Kaswan, J. S. Dhatterwal, K. Malik, and A. Baliyan, in 2023 International Conference on Communication, Security and Artificial Intelligence (ICCSAI) (2023), pp. 699–704.

12. G. Brunner, A. Konrad, Y. Wang, and R. Wattenhofer, Midi-vae: Modeling dynamics and instrumentation of music with applications to style transfer (2018), URL https://api.semanticscholar.org/CorpusID:49317433.

13. T. Karras, S. Laine, and T. Aila, A style-based generator architecture for generative adversarial networks (2021), URL https://doi.org/10.1109/TPAMI.2020.2970919.

14. D. Jonassen, J. Howland, J. L. Moore, and R. Marra, in Learning to Solve Problems with Technology: A Constructivist Perspective (2002).

15. E. D. Liddy, Natural language processing. (2001).

16. D. Jatnika, M. A. Bijaksana, and A. A. Suryani, Word2vec model analysis for semantic similarities in english words (2019), the 4th International Conference on Computer Science and Computational Intelligence (ICCSCI 2019) : Enabling Collaboration to Escalate Impact of Research Results for Society, URL https://www.sciencedirect.com/science/article/ pii/S1877050919310713.

17. Venkatesh, S. U. Hegde, Z. A. S, and N. Y, in 2021 International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) (2021), pp. 1–8.

18. L. Wang, Y. Ling, Z. Yuan, M. Shridhar, C. Bao, Y. Qin, B. Wang, H. Xu, and X. Wang, Gensim: Generating robotic simulation tasks via large language models (2024), 2310.01361.

19. A. Namboori, S. Mangale, A. Rosenbaum, and S. Soltan, Gemquad : Generating multilingual question answering datasets from large language models using few shot learning (2024), 2404.09163.

20. T. Mikolov, M. Karafiat, L. Burget, J. Cernocky´, and S. Khudanpur, in Recurrent neural network based language model (2010), vol. 2, pp. 1045–1048.

21. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Cernocky, Strategies for training large scale neural network language models (2011).

22. S. Islam, H. Elmekki, A. Elsebai, J. Bentahar, N. Drawel, G. Rjoub, and W. Pedrycz, A comprehensive survey on applications of transformers for deep learning tasks (2023), 2306.07303.

23. P. Yin and G. Neubig, in A Syntactic Neural Model for General-Purpose Code Generation (2017), pp. 440–450.

24. B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, Information and Software Technology 51, 7 (2009), ISSN 0950-5849, special Section – Most Cited Articles in 2002 and Regular Research Papers, URL http: //www.sciencedirect.com/science/article/pii/S0950584908001390.

25. K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, in Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (BCS Learning and Development Ltd., 2008), EASE’08, p. 68–77.

26. N. S. M. Yusop, J. Grundy, and R. Vasa, IEEE Transactions on Software Engineering 43, 848 (2017).

27. R. Moguel-S´anchez, C. Mart´ınez-Palacios, J. Ochar´an-Hern´andez, X. Lim´on, and A. S´anchez-Garc´ıa, Programming and Computer Software 49, 712 (2024).

28. P. O. Silva-Vasquez, V. Y. Rosales-Morales, and E. Ben´ıtez-Guerrero, Program. Comput. Softw. 48, 685–701 (2022), ISSN 0361-7688, URL https://doi.org/10.1134/S0361768822080187.

29. U. Dissemination, The database of abstracts of reviews of effects (dare) (2002).

30. P. Archana, P. B. Harish, N. Rajan, S. P, and N. S. Kumar, in 2021 Asian Conference on Innovation in Technology (ASIANCON) (2021), pp. 1–8.

31. D. Wang, W. Dong, and Y. Zhang, in 2020 35th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) (2020), pp. 98–104.

32. H. Phan, A. Sharma, and A. Jannesari, in 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) (2021), pp. 219–226.

33. Z. Nan, H. Guan, and X. Shen, HISyn: Human Learning-Inspired Natural Language Programming (Association for Computing Machinery, New York, NY, USA, 2020), p. 75–86, ISBN 9781450370431, URL https://doi.org/10.1145/3368089. 3409673.

34. Z. Zhang, S. Wu, R. Jiang, M. Pan, and T. Zhang, in Proceedings of the Tenth Asia-Pacific Symposium on Internetware (Association for Computing Machinery, New York, NY, USA, 2018), Internetware ’18, ISBN 9781450365901, URL https: //doi.org/10.1145/3275219.3275229.

35. Z. Nan, H. Guan, X. Shen, and C. Liao, in Proceedings of the 30th ACM SIGPLAN International Conference on Compiler Construction (Association for Computing Machinery, New York, NY, USA, 2021), CC 2021, p. 141–152, ISBN 9781450383257, URL https://doi.org/10.1145/3446804.3446852.

36. I. Gavran, E. Darulova, and R. Majumdar, Interactive synthesis of temporal specifications from examples and natural language (2020), URL https://doi.org/10.1145/3428269.

37. S. Zamanirad, B. Benatallah, M. C. Barukh, F. Casati, and C. Rodriguez, in 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) (2017), pp. 832–837.

38. H. Guan, X. Shen, and H. Krim, in SC17: International Conference for High Performance Computing, Networking, Storage and Analysis (2017), pp. 1–14.

39. T. Gvero and V. Kuncak, in 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering (2015), vol. 2, pp. 689–692.

40. N. Jain, S. Vaidyanath, A. Iyer, N. Natarajan, S. Parthasarathy, S. Rajamani, and R. Sharma, in 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) (2022), pp. 1219–1231.

41. N. Tao, A. Ventresque, and T. Saber, in 2023 IEEE Latin American Conference on Computational Intelligence (LA-CCI) (2023), pp. 1–6.

42. S. Lee, S. Y. Nam, and J. Kim, Program synthesis through learning the input-output behavior of commands (2022).

43. N. Tao, A. Ventresque, and T. Saber, in Optimization and Learning – 5th International Conference, OLA 2022, Syracuse, Sicilia, Italy, July 18-20, 2022, Proceedings, edited by B. Dorronsoro, M. Pavone, A. Nakib, and E.-G. Talbi (Springer, 2022), vol. 1684 of Communications in Computer and Information Science, pp. 240–252, ISBN 978-3-031-22039-5, URL https://doi.org/10.1007/978-3-031-22039-5_19.

44. A. Bhardwaj, P. Khanna, S. Kumar, and Pragya, Generative model for nlp applications based on component extraction (2020), international Conference on Computational Intelligence and Data Science, URL https://www.sciencedirect. com/science/article/pii/S1877050920308577.

45. K. Rahmani, M. Raza, S. Gulwani, V. Le, D. Morris, A. Radhakrishna, G. Soares, and A. Tiwari, Multi-modal program inference: a marriage of pre-trained language models and component-based synthesis (2021), URL https://doi.org/10. 1145/3485535.

46. P. Denny, J. Leinonen, J. Prather, A. Luxton-Reilly, T. Amarouche, B. A. Becker, and B. N. Reeves, in Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (Association for Computing Machinery, New York, NY, USA, 2024), SIGCSE 2024, p. 296–302, ISBN 9798400704239, URL https://doi.org/10.1145/3626252.3630909.

47. M. Kacmajor and J. D. Kelleher, Automatic acquisition of annotated training corpora for test-code generation (2019), URL https://www.mdpi.com/2078-2489/10/2/66.

48. M.-F. Wong, S. Guo, C.-N. Hang, S.-W. Ho, and C.-W. Tan, Natural language generation and understanding of big code for ai-assisted programming: A review (2023), URL https://www.mdpi.com/1099-4300/25/6/888.

49. C. E. A. Coello, M. N. Alimam, and R. Kouatly, Effectiveness of chatgpt in coding: A comparative analysis of popular large language models (2024), URL https://www.mdpi.com/2673-6470/4/1/5.

50. M. Rabinovich, M. Stern, and D. Klein, Abstract syntax networks for code generation and semantic parsing (2017), URL https://arxiv.org/abs/1704.07535.