Automated Object Storage Management Approach with Operator SDK and Custom Resource Definition

The work is devoted to the study of automation tools for managing stateful applications in the Kubernetes environment, particularly object storage systems. A review of existing management tools capable solving the set tasks is made. A comparative description of the considered tools based on review is given and a tool is selected that meets the introduced criteria: popularity, support form Kubernetes, reactivity of developed operator, additional features, and others. An approach to automatic object storage management using the Operator SDK and Custom Resource Definition is suggested. As a result of comprehensive comparative analysis of tools Kubebuilder, Juju, Metacontroller, Kudo and Operator SDK, the last one was chosen as a base of approach implementation. The architecture of the system for managing a containerized version of storage systems based on the Kubernetes platform and integrating the operator with a user monitoring system is proposed. The described approach is implemented in a software tool – an operator of the object data storage system resource. The paper describes the details of software implementation, the structure of the storage custom resource descriptor, and methods for testing the end system. As a result, an object storage management system based on the Kubernetes platform was created, which made it possible to reduce both labor costs for supporting and maintaining the system, and it’s cost by reducing dependence on hardware. Moreover, described approach corresponds to such features of modern object storages as multi-tier, erasure coding support, geo-replication, cluster topology that is quite innovative among existing automated storage management approaches on Kubernetes platforms.

1. . J. Gantz, D. Reinsel. The Digital Universe in 2020: Big Data, Bigger Digital Shadow s, and Biggest Growth in the Far East. IDC IVIEW [Online]. Available: https://www.cs.princeton.edu/courses/archive/spring13/cos598C/idc-the-digital-universe-in-2020.pdf [Accessed Mar. 24, 2022].

2. . International Data Corporation (IDC), “Changing the way the world thinks about the impact of technology on business and society”. [Online]. Available: https://www.idc.com/ [Accessed Mar. 24, 2022].

3. . V. Yu. Shevtsov, E. S. Abramov, “The Analysis of Modern Data Storage Systems”, NBI tehnologii, 2019, vol. 13, no. 1, pp. 25-30 (in Russian).

4. . R. Dua, A. R. Raja, D. Kakadia. Virtualization vs containerization to support PaaS, in: Proc. of 2014 IEEE Int'l Conf. on Cloud Engineering, IC2E'14, 2014, pp. 610 - 614.

5. . S. M. Nanyan, T. N. Nichushkina, “Virtual Docker containers: purpose and application features”, Inzhenernyj Vestnik, 2015, no. 2, p. 2 (in Russian).

6. . J. Turnbull. The Docker Book, 2014, p. 338. [E-book] Available: https://dockerbook.com/.

7. . D. Silakov, “Docker project. Manage virtual environments”, Sistemnyj administrator, 2015, no. 3, pp. 10-14 (in Russian).

8. . D. Silakov, “Tools to manage multiple Docker containers”, Sistemnyj administrator, 2015, no. 5 (150), pp. 11-15 (in Russian).

9. . T. Uphill, J. Arundel, N. Khare, H. Saito, H.-C. Chloe Lee, Ke-J. Carol Hsu, DevOps: Puppet, Docker, and Kubernetes. Packt Publishing, 2017.

10. . Z. A. Orlov, “Study of scaling tools for container virtualization systems”, Alleya nauki, 2017, vol. 2, no. 9, pp. 867-871 (in Russian).

11. . M. Luksha, Kubernetes in action. DMK Press, 2018 (in Russian).

12. . S. Yaremchuk, “Get to know Kubernetes”, Sistemnyj administrator, 2017, no. 1-2 (170-171), pp. 41-45 (in Russian).

13. . J. Shah, D. Dubaria, “Building modern clouds: Using docker, kubernetes google cloud platform”, in: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, 2019, pp. 184-189.

14. . D. Safronov, K. M. Stonozhenko, I. V. Nikiforov, “Automatic load balancing between streaming data processing and cluster internal tasks using Kubernetes”, in: Modern technologies in programming theory and practice, Peter the Great St. Petersburg Polytechnic University; Dell Technologies; EPAM Systems, Polytech-Press, 2020, pp. 165-167 (in Russian).

15. . Kubernetes Deployment vs. StatefulSets. [Online] Available: https://www.baeldung.com/ops/kubernetes-deployment-vs-statefulsets [Accessed Mar. 25, 2022].

16. . Operator pattern. [Online] Available: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/ [Accessed Mar. 25, 2022].

17. . P. S. P. Shenoy, S. S. Vishnu, R. P. Kumar, S. Bailuguttu, “Enhancement of observability using Kubernetes operator”, Indonesian Journal of Electrical Engineering and Computer Science, 2022, vol. 25, no. 1, pp. 496-503.

18. . D. Silakov, “JUJU project. Deploy complex applications with one click”, Sistemnyj administrator, 2015, no. 10 (155), pp. 4-8 (in Russian).

19. . Juju SDK Documentation. [Online] Available: https://juju.is/docs/sdk [Accessed Mar. 25, 2022].

20. . Introduction to Metacontroller. [Online] Available: https://metacontroller.github.io/metacontroller/intro.html [Accessed Mar. 25, 2022].

21. . B. Shrishail, S. Ashish, G. Sagar, J. Chinmay. “A QOS-aware secure personal cloud storage with ubiquitous access and smart home extension”, in: 2015 International Conference on Computer, Communication and Control (IC4), 2015.

22. . C. Wu, V. Sreekanti, J. M. Hellerstein. “Eliminating Boundaries in Cloud Storage with Anna”, CoRR, 2018, vol. 1809.00089.

23. . Dell EMC ECS: powering a data-driven future. [Online] Available: https://www.dell.com/en-us/dt/learn/data-storage/ecs.htm [Accessed Mar. 25, 2022].

24. . A. S. Shemyakinskaya, I. V. Nikiforov, Hard drives monitoring automation approach for Kubernetes container orchestration system // Proceedings of the Institute for System Programming of the RAS. – 2020. – Vol. 32. – No 2. – P. 99-106. – DOI 10.15514/ISPRAS-2020-32(2)-8.

25. . Extend the Kubernetes API with CustomResourceDefinitions. [Online] Available: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/ [Accessed Mar. 29, 2022].

26. . S. Yaremchuk, “Deploy monitoring Prometheus + Grafana”, Sistemnyj administrator, 2017, no. 5 (174), pp. 36-44 (in Russian).