Continuous integrating modules of distributed applied software packages in Orlando Tools

We propose a new approach to solving important practical problems of complex debugging, joint testing, and analysis of the execution time of software module versions in a heterogeneous distributed computing environment that integrating Grid and cloud computing. These problems arise in the process of supporting the continuous integration of modules of distributed applied software packages. The study focuses on the packages that are used to conduct large-scale computational experiments. The scientific novelty of the proposed approach is to combine the methodology for creating the packages with modern software development practices based on its continuous integration using knowledge about the specifics of the problems being solved. Our contribution is multifold. We expanded the capabilities of continuous integration tools by developing new additional tools for the markup and transformation of data from poorly structured sources and predicting modules execution time. In addition, we developed a technological scheme of the joint applying our developed tools and external systems for continuous integration. Therefore, we provide a more large range of capabilities of continuous integration in relation to the processes of creating and using the packages in comparison with the well-known tools. The fundamental basis of their functioning is a new conceptual model of the packages. This model supports the specification, planning, and execution of software continuous integration processes taking into account the specific subject data and problems being solved. Applying the developed tools in practice leads to a decrease in the number of errors and failures of applied software in the development and use of the packages. In turn, such decrease significantly reduces the time for large-scale computational experiments and increases the efficiency of using resources of the environment. The results of practical experiments on the use of system prototype for continuous integration of applied software show their high efficiency.

1. Il’in V.P., Skopin I.N. About performance and intellectuality of supercomputer modeling. Programming and Computer Software, vol. 42, no. 1, 2016, pp. 5-16.

2. Massobrio R., Nesmachnow S., Tchernykh A., Avetisyan A., Radchenko G. Towards a Cloud Computing Paradigm for Big Data Analysis in Smart Cities. Programming and Computer Software, vol. 44, no. 3, 2018, pp. 181-189.

3. Deelman E., Peterka T., Altintas I., Carothers C.D., van Dam K.K., Moreland K., Parashar M., Ramakrishnan L., Taufer M., Vetter J. The future of scientific workflows. The International Journal of High Performance Computing Applications, vol. 32, no. 1, 2018, pp. 159 175.

4. Krol M., Rene S., Ascigil O., Psaras I. ChainSoft: Collaborative Software Development using Smart Contracts. In Proc. of the 1st Workshop on Cryptocurrencies and Blockchains for Distributed Systems, 2018, pp. 1-6.

5. Sochat V. Containershare: Open Source Registry to build, test, deploy with CircleCI. The Journal of Open Source Software, vol. 3, no. 28, 2018, pp. 1-3.

6. Soni M., Berg A.M. Jenkins 2.x Continuous Integration Cookbook. Packt Publishing, 2017, 438 p.

7. Machiraju S., Gaurav S. Deployment via TeamCity and Octopus Deploy. In S. Machiraju, S. Gaurav. DevOps for Azure Applications. Apress, 2018, pp. 11-38.

8. Beller M., Gousios G., Zaidman A. Oops, My Tests Broke the Build: An Explorative Analysis of Travis CI with GitHub. In Proc, of the 14th International Conference on Mining Software Repositories, 2017, pp. 356-367.

9. Gruver G. Start and Scaling Devops in the Enterprise. BookBaby, 2016, 100 p.

10. Shahin M., Babar M.A., Zhu L. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access, 2017, pp. 3909-3943.

11. Wolff E. A Practical Guide to Continuous Delivery. Addison-Wesley, 2017, 265 p.

12. CruiseControl.NET (online). Available at: http://sourceforge.net/projects/ccnet, accessed 10.12.2018.

13. Apache Gump (online). Available at: https://gump.apache.org, accessed 10.12.2018.

14. Conan C/C++ package manager (online). Available at: https://www.conan.io, accessed 10.12.2018.

15. BuildMaster (online). Available at: https://inedo.com/buildmaster, accessed 10.12.2018.

16. Heckel T. (2015) Meet Travis CI: Open Source Continuous Integration, InfoQ (online). Available at: https://www.infoq.com/news/2013/02/travis-ci, accessed 10.12.2018.

17. Chacon S., Straub B. Pro git. Apress, 2014, 419 p.

18. Barrett D., Silverman R., Byrnes R. SSH: The Secure Shell. O'Reilly, 2005, 672 p.

19. Blum R. Linux Command Line and Shell Scripting Bible, Wiley, 2017, 816 p.

20. Colvin H. VirtualBox: An Ultimate Guide Book on Virtualization with VirtualBox. CreateSpace Independent Publishing Platform, 2015, 70 p.

21. Крупин A. Обзор пакета Parallels Remote Application Server для виртуализации рабочих мест: все включено. 3DNews – Daily Digital Digest (online). Доступно по ссылке: https://3dnews.ru/931400, дата обращения 10.12.2018 / Krupin A. Overview of the Package Parallels Remote Application Server for Workplace Virtualization: All-inclusive, 3DNews – Daily Digital Digest, 2016. Available at: https://3dnews.ru/931400, accessed 10.12.2018 (In Russian).

22. Smith R. Docker Orchestration. Packt Publishing – ebooks Account, 2017, 284 p.

23. Luksa M. Kubernetes in Action. Manning Publications, 2018, 624 p.

24. Varlamov M.I., Turdakov D.Y. A survey of methods for the extraction of information from Web resources. Programming and Computer Software, vol. 42, no. 5, 2016, pp. 279 291.

25. Feoktistov A., Kostromin R., Sidorov I.A., Gorsky S.A. Development of Distributed Subject-Oriented Applications for Cloud Computing through the Integration of Conceptual and Modular Programming. In Proc. of the 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, 2018, pp. 256-261.

26. JFrog Artifactory (online). Available at: https://jfrog.com/artifactory, accessed 10.12.2018.

27. Debian Package Tracking System – Advanced Packaging Tool (online). Available at: https://packages.qa.debian.org/a/apt.html, accessed 10.12.2018.

28. Yum Package Manager (online). Available at: http://yum.baseurl.org, accessed 10.12.2018.

29. Bychkov I.V., Oparin G.A., Tchernykh A.N., Feoktistov A.G., Gorsky S.A., Rivera-Rodriguez R. Scalable Application for Searching Global Minimum of Multiextremal Functions. Optoelectronics, Instrumentation and Data Processing, vol. 54, no. 1, 2018, pp. 83-89.

30. Tannenbaum T., Wright D., Miller K., Livny M. Condor – A Distributed Job Scheduler. In Beowulf Cluster Computing with Linux, The MIT Press, 2002, pp. 307-350.

31. Иркутский суперкомпьютерный центр СО РАН (online). Доступно по ссылке: http://hpc.icc.ru, дата обращения 10.12.2018 / Irkutsk Supercomputer center of SB RAS. Available at: http://hpc.icc.ru, accessed 10.12.2018.