Stateful Stream Processing Containerized as Microservice to Support Digital Twins in Fog Computing

Digital twins of processes and devices use information from sensors to synchronize their state with the entities of the physical world. The concept of stream computing enables effective processing of events generated by such sensors. However, the need to track the state of an instance of the object leads to the impossibility of organizing instances of digital twins as stateless services. Another feature of digital twins is that several tasks implemented on their basis require the ability to respond to incoming events at near-real-time speed. In this case, the use of cloud computing becomes unacceptable due to high latency. Fog computing manages this problem by moving some computational tasks closer to the data sources. One of the recent solutions providing the development of loosely coupled distributed systems is a Microservice approach, which implies the organization of the distributed system as a set of coherent and independent services interacting with each other using messages. The microservice is most often isolated by utilizing containers to overcome the high overheads of using virtual machines. The main problem is that microservices and containers together are stateless by nature. The container technology still does not fully support live container migration between physical hosts without data loss. It causes challenges in ensuring the uninterrupted operation of services in fog computing environments. Thus, an essential challenge is to create a containerized stateful stream processing based microservice to support digital twins in the fog computing environment. Within the scope of this article, we study live stateful stream processing migration and how to redistribute computational activity across cloud and fog nodes using Kafka middleware and its Stream DSL API.

1. A. Tchernykh, M. Babenko, N. Chervyakov et al. Scalable Data Storage Design for Non-Stationary IoT Environment with Adaptive Security and Reliability. IEEE Internet of Things Journal, vol. 7, no. 10, 2020, pp. 10171-10188.

2. G. E. Modoni, M. Sacco, W. Terkaj. A Telemetry-driven Approach to Simulate Data-intensive Manufacturing Processes. Procedia CIRP, vol. 57, 2016, pp. 281-285.

3. E. H. Glaessgen and D. D. S. Stargel. The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles. In Proc. of the 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference - Special Session on the Digital Twin, 2012, pp. 1-14.

4. M. Grieves and J. Vickers. Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems. In Transdisciplinary Perspectives on Complex Systems, Springer, 2017, pp. 85–113.

5. Q. Zhang, L. Cheng, R. Boutaba. Cloud computing: state-of-the-art and research challenges. Journal of Internet Services and Applications, vol. 1, no. 1, 2010, pp. 7-18.

6. Б.М. Шабанов, О.И. Самоваров. Принципы построения межведомственного центра коллективного пользования общего назначения в модели программно-определяемого ЦОД. Труды ИСП РАН, том 30, вып. 6, 2018 г., стр. 7-24. DOI: 10.15514/ISPRAS-2018-30(6)-1 / B.M. Shabanov and O.I. Samovarov. Building the Software-Defined Data Center. Programming and Computer Software, vol. 45, no. 8, 2019, pp. 458-466 (in Russian).

7. G. I. Radchenko, A. B. A. Alaasam, A. N. Tchernykh. Comparative Analysis of Virtualization Methods in Big Data Processing. Supercomputing Frontiers and Innovations, vol. 6, no. 1, 2019, pp. 48-79.

8. J. Luo, L. Yin, J. Hu et al. Container-based fog computing architecture and energy-balancing scheduling algorithm for energy IoT. Future Generation Computer Systems, vol. 97, 2019, pp. 50-60.

9. M. Aazam, S. Zeadally, K. A. Harras. Deploying Fog Computing in Industrial Internet of Things and Industry 4.0. IEEE Transactions on Industrial Informatics, vol. 14, no. 10, 2018, pp. 4674-4682.

10. S. Singh, A. Angrish, J. Barkley et al. Streaming Machine Generated Data to Enable a Third-Party Ecosystem of Digital Manufacturing Apps. Procedia Manufacturing, vol. 10, 2017, pp. 1020-1030.

11. Y. Qamsane, C. Chen, E. C. Balta et al. A Unified Digital Twin Framework for Real-time Monitoring and Evaluation of Smart Manufacturing Systems. In Proc. of the 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE), 2019, pp. 1394-1401.

12. M. Burgess. Locality, Statefulness, Causality in Distributed Information Systems Concerning the Scale Dependence Of System Promises. arXiv, 2019, available: http://arxiv.org/abs/1909.09357.

13. C. Peiffer and I. L’Heureux. System and method for maintaining statefulness during client-server interactions. United States Patent. US8346848B2, 2013.

14. M. Naseri and A. Towhidi. Stateful Web Services: A Missing Point in Web Service Standards. In In Proc. of the International MultiConference of Engineers and Computer Scientists, 2007, pp. 993-997.

15. R. Lichtenthaler. Model-driven software migration towards fine-grained cloud architectures. CEUR Workshop Proceedings, vol. 2339, 2019, pp. 35-38.

16. S. Newman. Building Microservices: Designing Fine-Grained System. O’Reilly Media, 2015, 280 p.

17. James Lewis and Martin Fowler. Microservices. 2014. Available at: https://martinfowler.com/articles/microservices.html, accessed Jan. 11, 2019.

18. The Microservice Revolution: Containerized Applications, Data and All, Available at: https://www.infoq.com/articles/microservices-revolution, accessed Dec. 10, 2019.

19. C. Fehling, F. Leymann, R. Retter, W. Schupeck, P. Arbitter. Cloud Computing Patterns: Fundamentals to Design, Build, Manage Cloud Applications. Vienna: Springer Vienna, 2014.

20. W. Li and A. Kanso. Comparing containers versus virtual machines for achieving high availability. In Proc. of the 2015 IEEE International Conference on Cloud Engineering, 2015, pp. 353-358.

21. C. Clark, K. Fraser, S. Hand et al. Live Migration of Virtual Machines. In Proc. of the 2nd conference on Symposium on Networked Systems Design & Implementation, vol. 2, 2005, pp. 273-286.

22. docker checkpoint | Docker Documentation, Available at: https://docs.docker.com/engine/reference/commandline/checkpoint, accessed Dec. 11, 2019.

23. A. Reber. CRIU and the PID dance. In Proc. of the Linux Plumbers Conference, 2019, pp. 1-4.

24. StatefulSets – Kubernetes, Available at: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#limitations, accessed Dec. 18, 2019.

25. L. Abdollahi Vayghan, M. A. Saied, M. Toeroe, F. Khendek. Microservice Based Architecture: Towards High-Availability for Stateful Applications with Kubernetes. In Proc. of the 19th IEEE International Conference on Software Quality, Reliability and Security, 2019, pp. 176–185.

26. H. Loo, A. Yeo, K. Yip, T. Liu. Live Pod Migration in Kubernetes. University of British Columbia, Vancouver, Canada. [Online]. Available at : https://www.cs.ubc.ca/~bestchai/teaching/cs416_2017w2/project2/project_m6r8_s8u8_v5v8_y6x8_proposal.pdf

27. H. Ohtsuji and O. Tatebe. Network-Based Data Processing Architecture for Reliable and High-Performance Distributed Storage System. Lecture Notes in Computer Science, vol. 9523, 2015, pp. 16–26.

28. A. B. A. Alaasam, G. Radchenko, A. Tchernykh. Stateful Stream Processing for Digital Twins: Microservice-Based Kafka Stream DSL. In Proc. of the International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), 2019, pp. 0804–0809.

29. A. Raddaoui, A. Settle, J. Garbutt, S. Singh. High Availability of Live Migration. openstack.org, 2017. [Online]. Available at: http://superuser.openstack.org/wp-content/uploads/2017/06/ha-livemigrate-whitepaper.pdf, accessed Dec. 11, 2019.

30. Д.А. Купляков, Е.В. Шальнов, В.С. Конушин, А.С. Конушин. Распределенный алгоритм сопровождения для подсчета людей в видео. Программирование, том. 45, no. 4, стр. 28-35. / D.A. Kuplyakov, E.V. Shalnov, V.S. Konushin, A.S. Konushin. A Distributed Tracking Algorithm for Counting People in Video. Programming and Computer Software, vol. 45, no. 4, 2019, pp. 163–170.

31. A. Sunderrajan, H. Aydt, A. Knoll. DEBS Grand Challenge : Real time Load Prediction and Outliers Detection using STORM. In Proc. of the 8th ACM International Conference on Distributed Event-Based Systems, 2014, pp. 294–297.

32. S. Trilles, S. Schade, O. Belmonte, J. Huerta. Real-Time Anomaly Detection from Environmental Data Streams. Lecture Notes in Geoinformation and Cartography, vol. 217, 2015, pp. 125–144.

33. Apache Kafka’s MirrorMaker – Confluent Platform, Available at: https://docs.confluent.io/4.0.0/multi-dc/mirrormaker.html, accessed Apr. 15, 2020.

34. G. Radchenko, A. Alaasam, A. Tchernykh. Micro-Workflows: Kafka and Kepler Fusion to Support Digital Twins of Industrial Processes. In Proc. of the IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), 2018, pp. 83–88.

35. А.Б.А. Алаасам, Г.И. Радченко, А. Н. Черных. Микро-потоки работ: сочетание потоков работ и потоковой обработки данных для поддержки цифровых двойников технологических процессов. Вестник Южно-Уральского государственного университета. Серия: Вычислительная математика и информатика, том 8, no. 4, 2019 г., стр. 100-116 / A. B. A. Alaasam, G. Radchenko, A. Tchernykh. Micro-Workflows: A Combination of Workflows and Data Streaming to Support Digital Twins of Production Processes. Bulletin of the South Ural State University. Series: Computational Mathematics and Software Engineering, vol. 8, no. 4, 2019, pp. 100–116, Nov. 2019 (in Russian).

36. A.B.A. Alaasam, G. Radchenko, A. Tchernykh et al. Scientific Micro-Workflows : Where Event-Driven Approach Meets Workflows to Support Digital Twins. In Proc. of the International Conference RuSCDays'18 – Russian Supercomputing Days, vol. 1, 2018, pp. 489–495.