The Software Implementation of a Metagraph Processing System Based on the Big Data Approach
https://doi.org/10.15514/ISPRAS-2022-34(1)-7
Abstract
The paper discusses the approach to solving the problem of processing metagraphs using Big Data technology. The formal definition of the metagraph data model and the metagraph agent model are given. The metagraph representation using the flat graph model is discussed. The flat graph and metagraph Big Data processing are described. The architecture of the system for processing data in metagraph is discussed. The metagraph processing using metagraph agents based on Big Data technology is discussed. The experiments result for parallel metagraph processing are given.
Supporting agencies: The article is based on the materials of the report at the Seventh International Conference «Actual Problems of System and Software Engineering» (APSSE 2021).
About the Authors
Valeriy Mikhailovich CHERNENKIY
Bauman Moscow State Technical University
Russian Federation
Russian Federation
Doctor of Technical Sciences, Professor
Ivan Vladimirovich DUNIN
Bauman Moscow State Technical University
Russian Federation
Russian Federation
Postgraduate Student
Yuriy Evegenievich GAPANYUK
Bauman Moscow State Technical University
Russian Federation
Russian Federation
Candidate of Technical Sciences, Associate Professor
References
1. Reut D., Falko S., Postnikova E. About scaling of controlling information system of industrial complex by streamlining of big data arrays in compliance with hierarchy of the present lifeworlds. International Journal of Mathematical, Engineering and Management Sciences, vol. 4, issue 5, 2019, pp. 1127-1139.
2. Chesnokov V. Overlapping community detection in social networks with node attributes by neighborhood influence. In Proc. of the 6th International Conference on Network Analysis, Springer Proceedings in Mathematics & Statistics, vol. 197, 2017, pp. 187-203.
3. Rasheed B., Popov A.Yu. Network graph datastore using DiSc processor. In Proc. of the 2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, 2019, pp. 1582-1587.
4. Abdymanapov C., Popov A.Yu. Motion planning algorithms using DISC. In: Proc. of the 2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, 2019, pp. 1844-1847.
5. Bozhko A.N. Hypergraph model for assembly sequence problem. In IOP Conference Series: Materials Science and Engineering, vol. 560, Issue 1, 2019.
6. Basu A., Blanning R. Metagraphs and Their Applications. Springer, 2007, 174 p.
7. Chernenkiy V.M., Gapanyuk Yu.E. et al. The Hybrid Multidimensional-Ontological Data Model Based on Metagraph Approach. Lecture Notes in Computer Science, vol. 10742, 2018, pp. 72-87.
8. Chernenkiy V.M., Gapanyuk Yu.E. et al. The Principles and the Conceptual Architecture of the Metagraph Storage System. Communications in Computer and Information Science, vol. 1003, 2018, pp. 73-87.
9. Malewicz G., Austern M.H. et al. Pregel: A System for Large-scale Graph Processing. In Proc. of the 2010 ACM SIGMOD International Conference on Management of Data, 2010, pp. 135-146.
10. Xin R.S., Crankshaw D. et al. GraphX: Unifying Data-Parallel and Graph-Parallel Analytics. arXiv preprint arXiv:1402.2394, 2014.
11. Shaposhnik R., Martella C., Logothetis D. Practical Graph Analytics with Apache Giraph. Apress, 2015, 334 p.
12. Introducing Gelly: Graph Processing with Apache Flink, URL:https://flink.apache.org/news/2015/08/24/introducing-flink-gelly.html, accessed 2021/04/09.
13. Faloutsos M., Faloutsos P., Faloutsos C. On power-law relationships of the internet topology. ACM SIGCOMM Computer Communication Review, vol. 29, issue 4, 1999, pp. 251-262.
14. Signal-Collect programming model, URL: https://uzh.github.io/signal-collect/documentation.html, accessed 2021/04/09.
15. Han M., Daudjee K. et al. An Experimental Comparison of Pregel-like Graph Processing Systems. Proceedings of the VLDB Endowment, vol. 7, issue 12, 2014, pp. 1047-1058.
16. Low Y., Gonzalez J. et al. Distributed GraphLab: A Framework for Machine Learning in the Cloud. arXiv preprint arXiv:1204.6078, 2012.
17. Gonzalez J.E., Low Y. et al. PowerGraph: Distributed graph-parallel computation on natural graphs. In Proc. of the 10th USENIX conference on Operating Systems Design and Implementation, 2012, pp. 17-30.
Review
For citations:
CHERNENKIY V.M., DUNIN I.V., GAPANYUK Yu.E. The Software Implementation of a Metagraph Processing System Based on the Big Data Approach. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2022;34(1):87-100. https://doi.org/10.15514/ISPRAS-2022-34(1)-7
Email this article 