Development and Use of Service-Oriented Scientific Applications in the FDE-SWFs Toolkit
https://doi.org/10.15514/ISPRAS-2024-36(6)-11
Abstract
The automation of workflow-based computing for solving large resource-intensive problems has undoubtedly had an impact on increasing the productivity of scientific research. In recent years, workflows have become the basis for abstractions covering data processing and high-performance computing using distributed applications. Workflow management systems are powerful tools for the collaborative development and use of distributed scientific applications. Nowadays, as part of the development of such systems, particular attention is currently being paid to supporting service-oriented scientific applications. Within this field of research, there is a large spectrum of problems related to the support of modular scientific applications, the standardization of their components and interfaces, the use of heterogeneous information and computing resources, and organization interdisciplinary research. Unfortunately, the solution to the above listed problems has not been fully implemented in known workflow management systems that support the development and use of service-oriented scientific applications. In this context, the paper discusses relevant aspects of organizing service-oriented computing in a computing environment with heterogeneous resources. The development of technologies for the development and use of service-oriented scientific applications, in which problem-solving schemes are formed in the form of workflows, is discussed. Existing standards for describing workflows are represented. A new framework for creating service-oriented scientific applications is proposed. It extends and complements the capabilities of systems for such purposes.
Keywords
heterogeneous distributed computing environment,
heterogeneous resources,
service-oriented scientific applications,
workflows.
Supporting agencies: The study was supported by the Ministry of Science and Higher Education of the Russian Federation, project no. № FWEW-2021-0005 «Technologies for the development and analysis of subject-oriented intelligent group control systems in non-deterministic distributed environments».
About the Authors
Alexander Gennadevich FEOKTISTOV
Matrosov Institute for System Dynamics and Control Theory of the Siberian Branch of the Russian Academy of Sciences
Russian Federation
Russian Federation
Dr. Sci. (Tech.), Associate Professor, Chief Researcher, Deputy Director of ISDCT SB RAS for scientific work. Fields of research: parallel and distributed computing, scientific applications, multi-agent technologies, simulation modeling, and testbeds.
Mikhail Leontevich VOSKOBOINIKOV
Matrosov Institute for System Dynamics and Control Theory of the Siberian Branch of the Russian Academy of Sciences
Russian Federation
Russian Federation
Junior researcher of ISDCT SB RAS. Research interests: parallel and distributed computing, service-oriented programming, and testbeds.
Andrei Nikolaevitch TCHERNYKH
Centro de investigación Científica y de educación Superior de Ensenada,
Ivannikov Institute for System Programming of the Russian Academy of Sciences
Mexico
Mexico
Dr. Sci (Phys.-Math.), Adjunct Chief Researcher at the ISP RAS, Full Professor in Computer Science at CICESE Research Center, Ensenada, Baja California, Mexico, Director of Parallel Computing Laboratory. Fields of research: multiobjective resource optimization in a cloud environment, both theoretical and experimental, security, scheduling with uncertainty, heuristics and meta-heuristics, adaptive resource allocation, and privacy-preserving machine learning with fully homomorphic encryption.
References
1. Erwin D.W., Snelling D.F. UNICORE: A Grid Computing Environment. Lecture Notes in Computer Science, 2001, vol. 2150, pp. 825-834. DOI: 10.1007/3-540-44681-8_116.
2. Litzkow M.J., Livny M., Mutka M.W. Condor – A hunter of idle workstations. In Proc. of the 8th International Conference on Distributed Computing Systems, Institute of Electrical and Electronics Engineers, 1988, pp. 104-111. DOI: 10.1109/DCS.1988.12507.
3. Deelman E., Vahi K., Juve G., Rynge M., Callaghan S., Maechling P. J., Mayani R., Chen W., da Silva R.F., Livny M., Wenger K. Pegasus, a workflow management system for science automation. Future Generation Computer Systems, 2015, vol. 46, pp. 17-35. DOI: 10.1016/j.future.2014.10.008.
4. Talia D. Workflow systems for science: Concepts and tools. International Scholarly Research Notices, 2013, vol. 2013, pp. 404525. DOI: 10.1155/2013/404525.
5. Da Silva R.F., Filgueira R., Pietri I., Jiang M., Sakellariou R., Deelman E. A characterization of workflow management systems for extreme-scale applications. Future Generation Computer Systems, 2017, vol. 75, pp. 228-238. DOI: 10.1016/j.future.2017.02.026.
6. Brown A., Johnston S., Kelly K. Using service-oriented architecture and component-based development to build web service applications. Rational Software Corporation, 2002, vol. 6. 16 p.
7. Afgan E., Baker D., Coraor N., Chapman B., Nekrutenko A., Taylor J. Galaxy CloudMan: delivering cloud compute clusters. BMC bioinformatics, 2010, vol. 11, no. 12, pp. 1-6. DOI: 10.1186/1471-2105-11-S12- S4.
8. Balis B. HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows. Future Generation Computer Systems, 2016, vol. 55, pp. 147-162. DOI: 10.1016/j.future.2015.08.015.
9. Hilman M.H., Rodriguez M.A., Buyya R. Workflow-as-a-service cloud platform and deployment of bioinformatics workflow applications. In Knowledge Management in the Development of Data-Intensive Systems (In Mistrik I., Galster M., Maxim B., Tekiner-dogan B. editors). Boca Raton, FL, USA, CRC Press, 2021, pp. 205-226.
10. Papazoglou M. Web Services: Principles and Technology. New York, Pearson Education, 2008. 752 p.
11. Welke R., Hirschheim R., Schwarz A. Service-oriented architecture maturity. Computer, 2011, vol. 44, no. 2, pp. 61-67. DOI: 10.1109/MC.2011.56.
12. Tsalgatidou A., Pilioura T. An overview of standards and related technology in web services. Distributed and Parallel Databases, 2002, vol. 12, pp. 135-162. DOI: 10.1023/A:1016599017660.
13. Ananthakrishnan R., Chard K., Foster I., Tuecke S. Globus platform‐as‐a‐service for collaborative science applications. Concurrency and Computation: Practice and Experience, 2015, vol. 27, no. 2, pp. 290-305. DOI: 10.1002/cpe.3262.
14. Foster I. Globus Online: Accelerating and democratizing science through cloud-based services. IEEE Internet Computing, 2011, vol. 15, no. 3, pp. 70-73. DOI: 10.1109/MIC.2011.64.
15. Foster I. Globus toolkit version 4: Software for service-oriented systems. Journal of Computer Science and Technology, 2006, vol. 21, pp. 513-520. DOI: 10.1007/s11390-006-0513-y.
16. Foster I., Kesselman C. The Grid: Blueprint for a new computing infrastructure. Morgan-Kaufmann, 2002. 667 p.
17. Foster I., Kesselman C. Globus: A metacomputing infrastructure toolkit. International Journal of Supercomputer Applications and High Performance Computing, 1997, vol. 11, no. 2, pp. 115-128. DOI: 10.1177/109434209701100205.
18. Juric M.B., Chandrasekaran S., Frece A., Hertis M., Srdic G. WS-BPEL 2.0 for SOA Composite applications with oracle SOA Suite 11g. Packt Publishing Ltd, 2010. 644 p.
19. Kim S., Bastani F.B., Yen I.L., Chen I.-R. High-assurance synthesis of security services from basic microservices. In Proc. of the 14th Intern. Symposium on Software Reliability Engineering (ISSRE 2003), IEEE, 2003, pp. 154-165. DOI: 10.1109/ISSRE.2003.1251039.
20. Fielding R.T. Architectural styles and the design of network-based software architectures. University of California, Irvine, 2000. 180 p.
21. Rajasekar A. iRODS primer: integrated rule-oriented data system. Morgan & Claypool Publishers, 2010. 143 p.
22. Sukhoroslov O. Building web-based services for practical exercises in parallel and distributed computing. Journal of Parallel and Distributed Computing, 2018, vol. 118, pp. 177-188. DOI: https://doi.org/10.1016/j.jpdc.2018.02.024.
23. Savchenko D.I., Radchenko G.I., Taipale O. Microservices validation: Mjolnirr platform case study. In Proc. of the 38th International convention on information and communication technology, electronics and microelectronics (MIPRO), IEEE, 2015, pp. 235-240. DOI: 10.1109/MIPRO.2015.7160271.
24. Бухановский А.В., Ковальчук С.В., Марьин С.В. Интеллектуальные программные комплексы компьютерного моделирования сложных систем: концепция, архитектура и примеры реализации. Известия высших учебных заведений. Приборостроение, 2009 г., том 52, № 10, стр. 5-24. (in Russian).
25. Васильев В.Н., Князьков К.В., Чуров Т.Н., Насонов Д.А., Марьин С.В., Ковальчук С.В., Бухановский А.В. CLAVIRE: облачная платформа для обработки данных больших объемов. Информационно-измерительные и управляющие системы, 2012, том 10, вып. 11, стр. 7-16. (in Russian).
26. Поляков С.В., Выродов А.В., Пузырьков Д.В., Якобовский М.В. Облачный сервис для решения многомасштабных задач нанотехнологии на суперкомпьютерных системах. Труды ИСП РАН, 2015, том 27, вып. 6, стр. 409-420. (in Russian). DOI 10.18522/2311-3103-2016-12-103114.
27. Кудрявцев А.О., Кошелев В.К., Избышев А.О., Дудина И.А., Курмангалеев Ш.Ф., Аветисян А.И., Иванников В.П., Велихов В.Е., Рябинкин Е.А. Разработка и реализация облачной системы для решения высокопроизводительных задач. Труды ИСП РАН, 2013, том 24, стр. 13-34.
28. Ханчук А.И., Сорокин А.А., Смагин С.И., Королев С.П., Макогонов С.В., Тарасов А.Г., Шестаков Н.В. Развитие информационно-телекоммуникационных систем в ДВО РАН. Информационные технологии и вычислительные системы, 2013, № 4, стр. 45-57.
29. Шокин Ю.И., Федотов А.М., Жижимов О.Л. Технологии создания распределенных информационных систем для поддержки научных исследований. Вычислительные технологии, 2015, том. 20, № 5, стр. 251-274.
30. Брагинская Л.П., Григорюк А.П., Ковалевский В.В., Загорулько Г.Б. Разработка научной среды для комплексных исследований в активной сейсмологии. Информационные технологии в науке, образовании и управлении: Материалы XLIV междунар. конф. и XIV междунар. конф. молодых учёных IT + S&E`16 (22 мая – 01 июня 2016 г., г. Гурзуф, Россия), М., Институт новых информационных технологий, 2016, стр. 3-10.
31. Массель Л.В., Болдырев Е.А., Макагонова Н.Н., Копайгородский А.Н., Черноусов А.В. ИТ-инфраструктура научных исследований: методический подход и реализация. Вычислительные технологии, 2006, том 11, № S8, стр. 59-68.
32. Бычков И.В., Ружников Г.М., Фёдоров Р.К., Шумилов А.С. Система планирования и выполнения композиций веб-сервисов в гетерогенной динамической среде. Вычислительные технологии, 2016, том 21, № 6, стр. 18-35.
33. Бычков И.В., Опарин Г.А., Феоктистов А.Г., Богданова В.Г., Пашинин А.А. Сервис-ориентированное управление распределенными вычислениями на основе мультиагентных технологий. Известия Южного федерального университета. Технические науки, 2014, № 12, стр. 17-27.
34. Бычков И.В., Опарин Г.А., Феоктистов А.Г., Богданова В.Г., Корсуков А.С. Сервис-ориентированный подход к организации распределённых вычислений с помощью инструментального комплекса DISCENT. Информационные технологии и вычислительные системы, 2014, № 2, стр. 7–15.
35. Kostromin R., Basharina O., Feoktistov A., Sidorov I. Microservice-Based Approach to Simulating Environmentally-Friendly Equipment of Infrastructure Objects Taking into Account Meteorological Data. Atmosphere, 2021, vol. 12, no. 9, pp. 1217. DOI: 10.3390/atmos12091217.
36. Михеев А., Орлов М. Война стандартов в мире workflow. Режим доступа: https://ecm-journal.ru/material/Vojjna-standartov-v-mire-workflow (дата обращения: 03.05.2024).
37. Артамонов И.В. Описание бизнес-процессов: вопросы стандартизации. Прикладная информатика, 2011, № 3 (33), стр. 20-28.
38. Web Services Business Process Execution Language Version 2.0. Available at: https://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.pdf, accessed 03.05.2024.
39. Common Workflow Language (CWL). Available at: https://www.commonwl.org, accessed 03.05.2024.
40. Feoktistov A., Edelev A., Tchernykh A., Gorsky S., Basharina O., Fereferov E. An Approach to Implementing High-Performance Computing for Problem Solving in Workflow-based Energy Infrastructure Resilience Studies. Computation, 2023, vol. 11, no. 12, pp. 243. DOI: 10.3390/computation11120243.
41. Feoktistov A., Gorsky S., Sidorov I., Bychkov I., Tchernykh A., Edelev A. Collaborative Development and Use of Scientific Applications in Orlando Tools: Integration, Delivery, and Deployment. Communications in Computer and Information Science, 2020, vol. 1087, pp. 18-32. DOI: 10.1007/978-3-030-41005-6_2.
42. Apache Airflow. Available at: https://airflow.apache.org/, accessed 03.05.2024.
43. Феоктистов А.Г., Костромин Р.О., Воскобойников М.Л., Ли-Дэ Д.И. Организация вычислительной среды разработки и применения научных рабочих процессов на основе контейнеризации. Вычислительные технологии, 2023, том 28, № 6, стр. 151-164. DOI: 10.25743/ICT.2023.28.6.013.
44. Danilov G., Voskoboinikov M. Testbed-based approach to testing a library for evaluating network reliability algorithms. In Proc. of the International Workshop on Critical Infrastructures in the Digital Worl (IWCI-2024), March 14-20, 2024, Bolshoe Goloustno, Russia, Irkutsk, ESI SB RAS, 2024, pp. 3-4.
45. Феоктистов А.Г., Воскобойников М.Л., Еделев А.В. Методы и средства разработки и применения испытательного стенда сервис-ориентированных приложений. Материалы VI Международного семинара по информационным, вычислительным и управляющим системам для распределенных сред (ICCS-DE 2024, 1-5 июля 2024 г., г. Иркутск, Россия), Иркутск, Изд-во ИДСТУ СО РАН, 2024, стр. 164-168.
Review
For citations:
FEOKTISTOV A., VOSKOBOINIKOV M., TCHERNYKH A. Development and Use of Service-Oriented Scientific Applications in the FDE-SWFs Toolkit. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2024;36(6):195-214. (In Russ.) https://doi.org/10.15514/ISPRAS-2024-36(6)-11
Email this article 