Typical HPC System Architecture for Numerical Simulation
Sergey Yurievich MOKSHIN,
Alexey Olegovich IGNATYEV,
Alexander Iosifovich MELNIKOV,
Dmitry Vladimirovich IVANKOV https://doi.org/10.15514/ISPRAS-2025-37(6)-34
Abstract
The paper discusses issues related to the topic of high-performance computing (HPC) systems designed to solve numerical simulation problems. A detailed description of the architecture, functional composition, as well as the system software used and information processing features in the HPC systems is provided. The authors of the work rely on their own long-term experience in creating HPC systems for Russian scientific and industrial centers. This work may be useful to specialists involved in the development and operation of modern high-performance computing systems designed for scientific research.
About the Authors
Sergey Yurievich MOKSHIN
Russian Federal Nuclear Center – Zababakhin All-Russian Research Institute of Technical Physics
Russian Federation
Russian Federation
Head of the Department of Zababakhin All-Russian Scientific Research Institute of Technical Physics since 2016. Research interests: design of supercomputer systems, development of functional subsystems for high performance supercomputing systems, operating systems development, methods and means for protecting information.
Alexey Olegovich IGNATYEV
Russian Federal Nuclear Center – Zababakhin All-Russian Research Institute of Technical Physics
Russian Federation
Russian Federation
Head of the Laboratory of Zababakhin All-Russian Scientific Research Institute of Technical Physics since 1998. Research interests: design of supercomputer systems, parallel numerical simulation programs development, operating systems development, methods and means of information security.
Alexander Iosifovich MELNIKOV
Russian Federal Nuclear Center – Zababakhin All-Russian Research Institute of Technical Physics
Russian Federation
Russian Federation
Leader scientist of Zababakhin All-Russian Scientific Research Institute of Technical Physics since 2015. Research interests: height performance computing, Linux development.
Dmitry Vladimirovich IVANKOV
Russian Federal Nuclear Center – Zababakhin All-Russian Research Institute of Technical Physics
Russian Federation
Russian Federation
Head of the Laboratory of Zababakhin All-Russian Scientific Research Institute of Technical Physics since 2016. Research interests: design of tiered data storage systems, development of high performance storage systems, research in data management methods.
References
1. J.E. Thornton, Design of a Computer – The Control Data 6600. In the editorials series of Malcolm C Harrison, Courant Institute of Mathematical Sciences, New York University, 1970
2. А. О. Лацис. Как построить и использовать суперкомпьютер. - М.: Бестселлер, 2003. -240 с. 3000 экз. ISBN 5-98158-003-8.
3. ГОСТ Р 57700.27-2020. ВВС. Термины и определения. 2020.
4. ГОСТ Р 57700.26-2020 Высокопроизводительные вычислительные системы. Требования приемочных испытаний.
5. Глазырин А.И., Мокшин С.Ю., Доклад «Суперкомпьютер «Зубр» средней производительности» // Всероссийская Конференция «Информационные технологии в оборонно-промышленном комплексе» (17-20 мая 2016 г), г. Челябинск.
6. Цифровые продукты РФЯЦ-ВНИИТФ. Available at: https://vniitf.ru/rubric/tsod-i-svem, accessed 10.05.2025.
7. Мокшин С.Ю. Общие подходы к проектированию подсистемы доступа высокопроизводительных вычислительных систем. Труды ИСП РАН, том 32 вып. 4, 2020, стр. 41-52, DOI: 10.15514/ISPRAS-2020-32(4)-3 / Mokshin S.Yu. General ways to design the access subsystem of high performance supercomputing systems. Trudy ISP RAN/Proc. ISP RAS, vol. 32, issue 4, 2020, pp. 41-52 (in Russia), DOI: 10.15514/ISPRAS-2020-32(4)-3.
8. Игнатьев А.О., Калинин А.А, Мокшин С.Ю. Реализация функций управления задачами и ресурсами высокопроизводительной вычислительной системы в «СПО Супер-ЭВМ». Труды ИСП РАН, том 34 вып. 2, 2022, стр. 159-178, DOI: 10.15514/ISPRAS-2022-34(2)-13 / Ignatyev A.O., Kalinin A.A., Mokshin S.Yu. Task and resources management function in HPC operation system «SPO Super-EVM». Trudy ISP RAN/Proc. ISP RAS, vol. 34, issue 2, 2022, pp. 159-178 (in Russia), DOI: 10.15514/ISPRAS-2022-34(2)-13.
9. Игнатьев А.О., Мокшин С.Ю., Иванков Д.В., Бекетов Е.А. Пути организации параллельного доступа к структурированным данным. Труды ИСП РАН, том 35 вып. 2, 2023, стр. 111-126, DOI: 10.15514/ISPRAS-2023-35(2)-8 / Ignatyev А.О., Mokshin S.Yu., Ivankov D.V., Beketov E.A. The parallel access to structured data organization. Trudy ISP RAN/Proc. ISP RAS, vol. 35, issue 2, 2023, pp. 111-126 (in Russia), DOI: 10.15514/ISPRAS-2023-35(2)-8.
10. Игнатьев А.О., Мокшин С.Ю. Типовая архитектура высокопроизводительной вычислительной системы для решения задач численного моделирования, Препринт РФЯЦ-ВНИИТФ № 265, Снежинск, 2020, 21 с. / Ignatyev A.O., Mokshin S.Yu. Base architecture of the mathematical modelling HPC system, Preprint FSUE «RFNC-VNIITF named after Academ. E.I. Zababakhin» № 265, Snezhinsk, 2020, 21 p. (in Russian).
11. TOP500. Available at: https://www.top500.org/, accessed 01.05.2025.
12. Understanding Lustre Filesystem Internals, Tech Report: ORNL/TM-2009/117, Available at: http://wiki.lustre.org/lid/index.html, accessed 10.05.2025.
13. BeeGFS, Available at: https://www.beegfs.io/, accessed 10.05.2025.
14. IBM Spectrum Scale. Available at: https://www.ibm.com/support/pages/ibm-spectrum-scale/, accessed 10.05.2025.
15. Ceph, Available at: https://ceph.io/, accessed 10.05.2025.
16. Open standard cloud computing platform. Available at: https://www.openstack.org/, accessed 10.04.2025.
17. ОСХД Стриж. Available at: https://vniitf.ru/data/marketing/ZOS/33.01-rukovodstvo%20programmista.pdf, accessed 10.04.2025.
18. High Performance Storage System. Available at: https://computing.llnl.gov/projects/hpss, accessed 10.04.2025
19. Enstore, Available at: https://github.com/Enstore-org/enstore/, accessed 10.05.2025.
20. АСХД. Программа для ЭВМ № 2018610434. Available at: https://onlinepatent.ru/software/2018610434/, accessed 10.04.2025.
21. MPI: The Message Passing Interface. Available at: http://parallel.ru/tech/tech_dev/mpi.html, accessed 01.05.2025.
22. The OpenMP API specification for parallel programming. Available at: https://www.openmp.org/, accessed 01.05.2025.
23. OpenLDAP. Available at: https://www.openldap.org/, accessed 01.05.2025.
24. Linaro DDT Debugger. Available at: https://www.linaroforge.com/linaro-ddt/, accessed 01.05.2025.
25. Perforce Totalview Debugger. Available at: https://totalview.io/, accessed 01.05.2025.
26. Salome. Available at: https://www.salome-platform.org/, accessed 01.05.2025.
27. VizIt. Available at: https://sd.llnl.gov/simulation/computer-codes/visit/, accessed 01.05.2025.
28. Paraview. Available at: https://www.paraview.org/, accessed 01.05.2025.
29. CentOS. Available at: https://www.centos.org/download/, accessed 01.05.2025.
30. Rocky Linux. Available at: https://rockylinux.org/, accessed 01.05.2025.
31. СПО Супер-ЭВМ. Available at: https://vniitf.ru/article/spo-super-evm, accessed 01.05.2025.
32. ЗОС Арамид. Available at: https://vniief.ru/researchdirections/civildevelopment/Aramid/, accessed 01.05.2025.
33. Eclipse. Available at: https://www.eclipse.org/, accessed 01.05.2025.
34. NetBeans. Available at: https://netbeans.apache.org/front/main/, accessed 01.05.2025.
35. Qt Creator. Available at: https://www.qt.io/product/development-tools, accessed 01.05.2025.
36. The Intel® oneAPI HPC Toolkit. Available at: https://www.intel.com/content/www/us/en/developer/tools/oneapi/hpc-toolkit.html, accessed 10.02.2022.
37. AMD ROCm. Available at: https://www.amd.com/en/products/software/rocm.html, accessed 10.04.2025.
38. Basic Linear Algebra Subprograms. Available at: https://netlib.sandia.gov/blas/, accessed 10.04.2025.
39. Linear Algebra PACKage. Available at: https://netlib.sandia.gov/lapack/, accessed 10.04.2025.
40. The Fastest Fourier Transform in the West (FFTW). Available at: https://fftw.org/, accessed 10.04.2025.
41. MPICH. Available at: https://www.mpich.org, accessed 30.05.2025.
42. MAVAPICH. Available at: https://mvapich.cse.ohio-state.edu/, accessed 10.04.2025.
43. OpenMPI. Available at: https://www.open-mpi.org/, accessed 10.04.2025.
44. Библиотека S-MPI. Available at: https://agora.guru.ru/abrau2013/pdf/338.pdf, accessed 10.04.2025.
45. Object Storage API. Swift documentation. Available at: https://docs.openstack.org/api-ref/ object-store/, accessed 10.05.2025.
46. Amazon S3 REST API Introduction – Amazon Simple Storage Service. Available at: https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html, accessed 10.05.2025.
47. ElasticSearch: The Official Distributed Search & Analytics Engine. Available at: https://elastic.co/elasticsearch/, accessed 10.05.2025.
48. Zabbix – open source distributed monitoring solution. Available at: https://zabbix.com/ru/, accessed 10.05.2025.
49. Highly Scalable Data Service (HSDS) – The HDF Group. Available at: https://hdfgroup.org/ solutions/highly-scalabale-data-service-hsds/, accessed 10.05.2025.
50. The HDF5 Library & File Format – The HDF Group. Available at: https://hdfgroup.org/ solutions/hdf5/, accessed 10.05.2025.
51. Иванков Д.В. Реализация функции долговременного хранения научных данных большого объема в вычислительном центре. Труды ИСП РАН, том 34 вып. 4, 2022, стр. 117-134. DOI: 10.15514/ISPRAS-2022-34(4)-9 / Ivankov D.V. Large-scale scientific data and long-term data storage function in a computing datacenter. Trudy ISP RAN/Proc. ISP RAS, vol. 34, issue 4, 2022. pp. 117-134 (in Russia), DOI: 10.15514/ISPRAS-2022-34(4)-9.
52. GPS. Available at: https://www.gps.gov/, accessed 10.05.2025.
53. ГЛОНАСС. Available at: https://glonass-iac.ru/en/about_glonass/, accessed 10.05.2025.
54. SVN. Available at: https://subversion.apache.org/, accessed 10.05.2025.
55. Git. Available at: https://git.kernel.org/pub/scm/git/git.git, accessed 10.05.2025.
56. Slurm Workload Manager. Available at: https://slurm.schedmd.com/, accessed 10.05.2025.
57. Ganglia. Available at: https://github.com/ganglia/, accessed 10.05.2025.
58. Nagios. Available at: https://www.nagios.org/, accessed 10.05.2025.
59. IPMI tool. Available at: https://github.com/ipmitool/ipmitool, accessed 10.05.2025.
60. PuTTY. Available at: https://www.putty.org/, accessed 10.05.2025.
61. Xming X Server. Available at: https://www.straightrunning.com/XmingNotes/, accessed 10.05.2025.
62. OpenText Exceed. Available at: www.opentext.com/products-and-solutions/products/specialty-technologies/connectivity/exceed/, accessed 11.12.2024.
63. TurboVNC. Available at: https://www.turbovnc.org/, accessed 10.05.2025.
64. Apache Software Foundation. Available at: https://www.apache.org/, accessed 10.05.2025.
65. Яндекс Браузер. Available at: https://browser.yandex.ru/, accessed 10.05.2025.
66. Postfix. Available at: https://www.postfix.org/, accessed 11.07.2023.
67. Dovecot. Available at: https://www.dovecot.org/, accessed 10.05.2025.
68. Jabberd2. Available at: https://jabberd2.org/, accessed 10.05.2025.
69. MariaDB Foundation. Available at: https://mariadb.org/, accessed 10.05.2025.
70. PostgreSQL. Available at: https://www.postgresql.org/, accessed 10.05.2025.
71. ANSYS. Available at: https://www.ansys.com/, accessed 19.02.2022.
72. SIEMENS. Available at: https://www.siemens.com/global/en/products/software.html, accessed 10.05.2025.
73. Dassault Systemes. Available at: https://dassault.fr/en/subsidiary/dassault-systemes, accessed 10.05.2025.
74. MathWorks. Available at: https://www.mathworks.com/, accessed 10.05.2025.
75. ЛОГОС. Available at: https://www.logos.vniief.ru/, accessed 10.05.2025.
76. OpenFOAM. Available at: https://www.openfoam.com/, accessed 10.05.2025.
77. LAMMPS. Available at: https://www.lammps.org/, accessed 18.02.2022.
Supplementary files
Review
For citations:
MOKSHIN S.Yu., IGNATYEV A.O., MELNIKOV A.I., IVANKOV D.V. Typical HPC System Architecture for Numerical Simulation. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2025;37(6):19-44. (In Russ.) https://doi.org/10.15514/ISPRAS-2025-37(6)-34
Email this article 