Mitigating Uncertainty in Developing Scientific Applications in Integrated Environment

The paper represents new means of the Orlando Tools framework. This framework is used as the basis of an integrated software environment for developing distributed applied software packages. The additional means are focused on mitigating various types of uncertainties arising from the job distribution in an integrated computing environment. They provide continuous integration, delivery, and deployment of applied and system software of packages. This helps to significantly reduce the negative impact of uncertainty on problem-solving time, computing reliability, and resource efficiency. An experimental analysis of the results of solving practical problems clearly demonstrates the advantages of applying these means.

1. Il’in V. Artificial Intelligence Problems in Mathematical Modeling. Communications in Computer and Information Science, vol. 1129, 2019, pp. 505-516.

2. Wang L., Jie W., Chen J. Grid computing: infrastructure, service, and applications. CRC Press, 2018, 528 p.

3. Varshney S., Sandhu R., Gupta P.K. QoS Based Resource Provisioning in Cloud Computing Environment: A Technical Survey. Communications in Computer and Information Science, vol. 1046, 2019, pp. 711-723.

4. Б.М. Шабанов, О.И. Самоваров. Принципы построения межведомственного центра коллективного пользования общего назначения в модели программно-определяемого ЦОД. Труды ИСП РАН, том 30, вып. 6, 2018 г., стр. 7-24. DOI: 10.15514/ISPRAS-2018-30(6)-1 / B.M. Shabanov and O.I. Samovarov. Building the Software-Defined Data Center. Programming and Computer Software, vol. 45, no. 8, 2019, pp. 458-466.

5. Mateescu G., Gentzsch W., Ribben C.J. Hybrid computing – where HPC meets grid and cloud computing // Future Generation Computer Systems, 2011, vol. 27, no. 5, pp. 440-453. DOI: 10.1016/j.future.2010.11.003.

6. Feoktistov A., Gorsky S., Sidorov I. et al. Orlando Tools: Energy Research Application Development through Convergence of Grid and Cloud Computing. Communications in Computer and Information Science, vol. 965, 2019, pp. 289-300.

7. Feoktistov A., Kostromin R., Sidorov I., Gorsky S. Development of Distributed Subject-Oriented Applications for Cloud Computing through the Integration of Conceptual and Modular Programming // In Proc. of the 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, 2018, pp. 256-261.

8. Yu J., Buyya R. A taxonomy of workflow management systems for grid computing. Journal of Grid Computing, vol. 3, no. 3-4, 2005, pp. 171-200.

9. Feoktistov A., Sidorov I., Tchernykh A. et al. Multi-Agent Approach for Dynamic Elasticity of Virtual Machines Provisioning in Heterogeneous Distributed Computing Environment In Proc. of the International Conference on High Performance Computing and Simulation (HPCS-2018), 2018, pp. 909-916.

10. Bychkov I., Oparin G., Feoktistov A. et al. Subject-oriented computing environment for solving large-scale problems of energy security research. Journal of Physics: Conference Series, vol. 1368, 2019, pp. 052030-1-052030-12.

11. Burri A., Dedner A., Klofkorn R., Ohlberger M. An efficient implementation of an adaptive and parallel grid in DUNE. Notes on Numerical Fluid Mechanics and Multidisciplinary Design, vol. 91, 2006, pp. 67-82.

12. Radchenko G., Hudyakova E. A service-oriented approach of integration of computer-aided engineering systems in distributed computing environments. In Proc. of the UNICORE Summit, 2012, pp. 57-66.

13. Shamakina A. Brokering service for supporting problem-oriented grid environments. In Proc. of the UNICORE Summit, 2012, pp. 67-75.

14. Bungartz H.J., Neumann P., Nagel W.E. Software for Exascale Computing-SPPEXA 2013-2015. Lecture Notes in Computational Science and Engineering, vol. 113, 2016, 565 p.

15. Afgan E., Baker D., Batut B. et al. The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2018 update. Nucleic Acids Research, vol. 46, no. W1, 2018, pp. W537-W544.

16. Ananthakrishnan R., Blaiszik B., Chard K., Chard R. Globus platform services for data publication. In Proc. of the Practice and Experience on Advanced Research Computing, 2018, pp. 1-7.

17. Sukhoroslov O. Supporting Efficient Execution of Workflows on Everest Platform. Communications in Computer and Information, vol. 1129, 2019, pp. 713-724.

18. Gavvala S.K., Chandrasheka J., Gangadharan G.R., Buyya R. QoS-aware cloud service composition using eagle strategy. Future Generation Computer Systems, vol. 90, 2019, pp. 273-290.

19. Deelman E., Peterka T., Altintas I., Carothers C.D. The future of scientific workflows. The International Journal of High Performance Computing Applications, vol. 32, no. 1, 2018, pp. 159-175.

20. Wangsom P., Lavangnananda K., Bouvry P. Multi-Objective Scientific-Workflow Scheduling with Data Movement Awareness in Cloud. IEEE Access, vol. 7, 2019, pp. 177063-177081.

21. Feoktistov A., Gorsky S., Sidorov I., Tchernykh A. Continuous Integration in Distributed Applied Software Packages. In Proc. of the 42st International Convention on Information and Communication Technology, Electronics and Microelectronics, 2019, pp. 1775-1780.

22. Gruver G. Start and Scaling Devops in the Enterprise. BookBaby, 2016. 100 p.

23. Talia D. Workflow Systems for Science: Concepts and Tools. ISRN Software Engineering, 2013, vol. 2013, Article ID 404525.

24. Deelman E., Vahi K., Juve G et al. Pegasus, a workflow management system for science automation // Future Generation Computer Systems, vol. 46, 2015, pp. 17-35.

25. Bumgardner V.K. OpenStack in Action. Manning Publications, 2016. 384 p.

26. Hirales-Carbajal A., González-García J. L., Tchernykh A. Workload Generation for Trace Based Grid Simulations. In Proc. of the 1st International Supercomputer Conference in Mexico (ISUM–2010), 2010, pp. 1-10.

27. Bychkov I., Oparin G., Tchernykh A. et al. Conceptual Model of Problem-Oriented Heterogeneous Distributed Computing Environment with Multi-Agent Management. Procedia Computer Science, vol. 103, 2017, pp. 162-167.

28. Соколинский Л.Б., Шамакина А.В. Методы управления ресурсами в проблемно-ориентированных вычислительных средах. Программирование, том 42, no. 1, 2016 г., стр. 26-38 / Sokolinsky L.B., Shamakina A.V. Methods of resource management in problem-oriented computing environment. Programming and Computer Software, vol. 42, no. 1, 2016, pp. 17-26.

29. Ramírez-Velarde R., Tchernykh A., Barba-Jimenez C. et al. Adaptive Resource Allocation with Job Runtime Uncertainty Journal of Grid Computing, vol. 15, no. 4, 2017, pp. 415-434.

30. Tchernykh A., Schwiegelshohn U., Talbi E.-G., Babenko M. Towards understanding uncertainty in cloud computing with risks of confidentiality, integrity, and availability. Journal of Computational Science, vol. 36, 2019, 100581. DOI: 10.1016/j.jocs.2016.11.011.

31. Kalyaev A.I., Kalyaev I.A. Method of multiagent scheduling of resources in cloud computing environments. Journal of Computer and Systems Sciences International, vol. 55, no. 2, 2016, pp. 211-221.

32. Bychkov I.V., Oparin G.A., Feoktistov A.G. et al. Multiagent control of computational systems on the basis of meta-monitoring and imitational simulation. Optoelectronics, Instrumentation and Data Processing, vol. 52, no. 2, 2016, pp. 107-112.

33. Java Agent DEvelopment Framework, available at: https://jade.tilab.com, accessed 30.05.2020.

34. Herrera J., Huedo E., Montero R., Llorente I. Porting of Scientific Applications to Grid Computing on GridWay. Scientific Programming, vol. 13, no. 4, 2005, pp. 317-331.

35. Tannenbaum T., Wright D., Miller K., Livny M. Condor – A Distributed Job Scheduler. In Beowulf Cluster Computing with Linux. The MIT Press, 2002, pp. 307-350.

36. Feoktistov A., Tchernych A., Kostromin R., Gorsky S. Knowledge Elicitation in Multi-Agent System for Distributed Computing Management. In Proc. of the 40th International Convention on Information and Communication Technology, Electronics and Microelectronics, 2017, pp. 1350-1355.

37. Feoktistov A., Kostromin R., Sidorov I. et al. Multi-Agent Algorithm for Re-Allocating Grid-Resources and Improving Fault-Tolerance of Problem-Solving Processes. Procedia Computer Science, vol. 150, 2019, pp. 171-178.

38. Vickrey W. Counterspeculation, Auctions, and Competitive Sealed Tenders. Journal of Finance, vol. 16, no. 1, 1961, pp. 8-37.