Систематический обзор литературы по тестированию программного обеспечения в контексте DevOps
https://doi.org/10.15514/ISPRAS-2023-35(1)-11
Аннотация
DevOps – это философия и инфраструктура, которые позволяют группам разработчиков и эксплуатации программного обеспечения работать скоординированно с целью быстрой и дешевой разработки и выпуска программного обеспечения. Однако, как сообщается в литературе, эффективность и преимущества DevOps зависят от нескольких факторов. В частности, было опубликовано несколько результатов исследований по автоматизации тестирования программного обеспечения, которая является краеугольным камнем фазы непрерывной интеграции в DevOps. Эти работы нуждаются в идентификации и классификации. В нашем исследовании консолидируется и классифицируется существующая литература по автоматизированному тестированию в контексте DevOps. Для исследования было проведено систематическое сопоставление литературных источников на основе 8 исследовательских вопросов. Путем выполнения запросов к шести уместным базам данных было получено 3312 статей. После процесса отбора 299 статей были выбраны в качестве основных. Исследователи сохраняют постоянный и растущий интерес к тестированию программного обеспечения в контексте DevOps. Большая часть исследований (71,2%) проводится в производственной сфере и затрагивают веб-приложения и SOA. Наиболее распространенными типами тестов являются модульные и интеграционные тесты.
При поддержке: Авторы признательны за отзывы членам Группы исследований и разработок в области программной инженерии Папского католического университета Перу.
Об авторах
Брайан ПАНДО
Национальный аграрный университет Ла-Сельвы
Перу
Перу
Компьютерный и системный инженер
Авраам ДАВИЛА
Папский католический университет Перу
Перу
Перу
Профессор программы компьютерной инженерии и докторант в области программной инженерии
Список литературы
1. Samarawickrama S.S., Perera I. Continuous scrum: A framework to enhance scrum with DevOps. In Proc. of the 17th International Conference on Advances in ICT for Emerging Regions, 2017, pp. 19-25.
2. Nicolau de França B.B., Jeronimo H., Travassos G.H. Characterizing DevOps by hearing multiple voices. In Proc. of the XXX Brazilian Symposium on Software Engineering, 2016, pp. 53-62.
3. Elliot S. DevOps and the Cost of Downtime: Fortune 1000 Best Practice Metrics Quantified. IDC, 2015, 13 p.
4. Ebert C., Gallardo G. et al. «DevOps», IEEE Software, vol. 33, issue 3, 2016, pp. 94-100.
5. Riungu-Kalliosaari L, Mäkinen S. et al. DevOps Adoption Benefits and Challenges in Practice: A Case Study. Lecture Notes in Computer Science, vol. 10027, 2016, pp. 590-597.
6. Stillwell M., Coutinho J.G.F. A DevOps approach to integration of software components in an EU research project. In Proc. of the 1st International Workshop on Quality-Aware DevOps, 2015, pp. 1-6.
7. Céspedes D., Angeleri P. et al. Software Product Quality in DevOps Contexts: A Systematic Literature Review. Advances in Intelligent Systems and Computing, vol. 1071, Springer, 2020, pp. 51-64.
8. Perera P., Silva R., Perera I. Improve software quality through practicing DevOps. In Proc. of the 17th International Conference on Advances in ICT for Emerging Regions, 2017, pp. 13-18.
9. Elberzhager F., Arif T. et al. From Agile Development to DevOps: Going Towards Faster Releases at High Quality – Experiences from an Industrial Context. Lecture Notes in Business Information Processing, vol. 269, 2017, pp. 33-44.
10. Jones S., Noppen J., Lettice F. Management challenges for devops adoption within UK SMEs. In Proc. of the 2nd International Workshop on Quality-Aware DevOps, 2016, pp. 7-11.
11. Soni M. End to End Automation on Cloud with Build Pipeline: The Case for DevOps in Insurance Industry, Continuous Integration, Continuous Testing, and Continuous Delivery. In Proc. of the IEEE International Conference on Cloud Computing in Emerging Markets, 2015, pp. 85-89.
12. Senapathi M., Buchan J., Osman H. DevOps capabilities, practices, and challenges: Insights from a case study. In Proc. of the International Conference on Evaluation and Assessment in Software Engineering, 2018, pp. 57-67.
13. Chen L. Continuous Delivery: Overcoming adoption challenges. Journal of Systems and Software, vol. 128, 2017, pp. 72-86.
14. Valdivia J.A., Lora-González A. et al. Patterns Related to Microservice Architecture: a Multivocal Literature Review. Programming and Computer Software, vol. 46, issue 8, 2020, pp. 594-608 / Вальдивия Х.А., Лора-Гонсалес А и др. Паттерны микросервисной архитектуры: многопрофильный обзор литературы. Труды ИСП РАН, том 33, вып. 1, 2021 г., стр. 81-96. DOI: 10.15514/ISPRAS-2021-33(1)-6.
15. Debois P. Agile Infrastructure & Operations. In Proc. of the Agile 2008 Conference, 2008, pp. 202-207.
16. Virmani M. Understanding DevOps & bridging the gap from continuous integration to continuous delivery. In Proc. of the 5th International Conference on Innovative Computing Technology, 2015, pp. 78-82.
17. Mullaguru S.N. Changing Scenario of Testing Paradigms using DevOps--A Comparative Study with Classical Models. Global Journal of Computer Science and Technology, vol. 15, issue 2, 2015, pp. 23-27.
18. Chernonozhkin S.K. Automated Test Generation and Static Analysis. Programming and Computer Software, vol. 27, issue 2, 2001, pp. 86-94 / Черноножкин С.К. Задача автоматического построения тестов и статистический анализ. Программирование, том 27, вып. 2, 2001 г., стр. 47-59.
19. Kuliamin V.V., Petrenko A.K. et al. The UniTesK Approach to Designing Test Suites. Programming and Computer Software, vol. 29, issue 6, 2003, pp. 310-322 / Кулямин В.В., Петренко А.К. и др. Подход UniTesK к разработке тестов. Программирование, том 29, вып. 6, 2003 г., стр. 25-43.
20. Jabbari R., Ali N., Petersen K. What is DevOps?: A Systematic Mapping Study on Definitions and Practices. In Proce. of the Scientific Workshop of XP2016, 2016, article no. 12, 11 p.
21. Ghantous G.B., Gill A. DevOps: Concepts, Practices, Tools, Benefits and Challenges. In Proc. of the 21st Pacific Asia Conference on Information Systems (PACIS), 2017, article no. 96, 13 p.
22. Lwakatare L.E., Kuvaja P., Oivo M. Dimensions of DevOps. Lecture Notes in Business Information Processing, vol. 212, 2015, pp. 212–217.
23. Katal A., Bajoria V., Dahiya S. DevOps: Bridging the gap between development and operations. In Proc. of the 3rd International Conference on Computing Methodologies and Communication, 2019, pp. 1-7.
24. Kamuto M.B., Langerman J.J. Factors inhibiting the adoption of DevOps in large organisations: South African context. In Proc. of the 2nd IEEE International Conference on Recent Trends in Electronics, Information and Communication Technology, 2017, pp. 48-51.
25. Zimmerer P. Strategy for Continuous Testing in iDevOps. In Proc. of the IEEE/ACM 40th International Conference on Software Engineering, 2018, pp. 532-533.
26. Fowler M. Continuous Integration. 2006. Available at: https://www.martinfowler.com/articles/continuousIntegration.html, accessed 28-nov-2020.
27. Parnin C., Helms E. et al. The Top 10 Adages in Continuous Deployment. IEEE Software, vol. 34, issue 3, 2017, pp. 86-95.
28. Fowler M. Continuous Delivery. 30-may-2013. Available at:: https://martinfowler.com/bliki/ContinuousDelivery.html, accessed 28-nov-2020.
29. Fitzgerald B., Stol K.J. Continuous software engineering and beyond: Trends and challenge. In Proc. of the 1st International Workshop on Rapid Continuous Software Engineering, 2014, pp. 1-9.
30. ISO/IEC/IEEE, «ISO/IEC/IEEE 24765:2017 Systems and software engineering – Vocabulary. Geneva, 2017.
31. Guide to the Software Engineering Body of Knowledge (SWEBOK), Version 3.0. IEEE Computer Society, 2014, 339 p.
32. Gupta R.K., Manikreddy P., Gv A. Challenges in adapting agile testing in a legacy product. In Proc. of the 11th IEEE International Conference on Global Software Engineering, 2016, pp. 104-108.
33. Jeeva Padmini K.V., Kankanamge P.S. et al. Challenges faced by agile testers: A case study. In Proc. of the Moratuwa Engineering Research Conference, 2018, pp. 431-436.
34. Coutinho J.C.S., Andrade W.L., Machado P.D.L. Requirements engineering and software testing in agile methodologies: A systematic mapping. In Proc. of the XXXIII Brazilian Symposium on Software Engineering, 2019, pp. 322-331.
35. Cruzes D.S., Moe N.B., Dyba T. Communication between developers and testers in distributed continuous agile testing. In Proc. of the 11th IEEE International Conference on Global Software Engineering, 2016, pp. 59-68.
36. Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, vol. 64, 2015, pp. 1-18.
37. Kuhrmann M., Diebold P., Münch J. Software process improvement: A systematic mapping study on the state of the art. PeerJ Computer Science, issue 5, 2016, article no. 62, 38 p.
38. Pinto G., Castor F. et al. Work practices and challenges in continuous integration: A survey with Travis CI users. Software: Practice and Experience, vol. 48, issue 12, 2018, pp. 2223-2236.
39. Shahin M., Ali Babar M., Zhu L. Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices. IEEE Access, vol. 5, 2017, pp. 3909-3943.
40. Alnafessah A., Gias A.U. et al. Quality-Aware DevOps Research: Where Do We Stand. IEEE Access, vol. 9, 2021, pp. 44476-44489.
41. Mascheroni M.A., Irrazábal E. Continuous Testing and Solutions for Testing Problems in Continuous Delivery: A Systematic Literature Review. Computación y Sistemas, vol. 22, issue 3, 2018, pp. 1009-1038.
42. Shahin M., Babar M.A. et al. Beyond Continuous Delivery: An Empirical Investigation of Continuous Deployment Challenges. In Proc. of the International Symposium on Empirical Software Engineering and Measurement, 2017, pp. 111-120.
43. Laukkanen E., Itkonen J., Lassenius C. Problems, causes and solutions when adopting continuous delivery – A systematic literature review. Information and Software Technology, vol. 82, 2017, pp. 55-79.
44. Sane P. A Brief Survey of Current Software Engineering Practices in Continuous Integration and Automated Accessibility Testing. In Proc. of the International Conference on Wireless Communications, Signal Processing and Networking, 2021, pp. 130-134.
45. Rajapakse R.N., Zahedi M. et al. Challenges and solutions when adopting DevSecOps: A systematic review. Information and Software Technology, vol. 141, 2021, article no. 106700, 27 p.
46. Daoudagh S., Lonetti F., Marchetti E. Continuous Development and Testing of Access and Usage Control. In Proc. of the European Symposium on Software Engineering, 2020, pp. 51-59.
47. Ampatzoglou A., Bibi S. et al. Identifying, categorizing and mitigating threats to validity in software engineering secondary studies», Information and Software Technology, vol. 106, 2019, pp. 201-230.
Рецензия
Для цитирования:
ПАНДО Б., ДАВИЛА А. Систематический обзор литературы по тестированию программного обеспечения в контексте DevOps. Труды Института системного программирования РАН. 2023;35(1):163-188. https://doi.org/10.15514/ISPRAS-2023-35(1)-11
For citation:
PANDO B., DÁVILA A. A Systematic Mapping Study on Software Testing in the DevOps Context. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2023;35(1):163-188. https://doi.org/10.15514/ISPRAS-2023-35(1)-11
Послать статью по эл. почте 