Designing Highly Loaded Systems

Nowadays, most of the services are moving online, which allows users to receive the service at any time. The high availability of the service leads to an increase in the number of users, which entails an increase in the load on the system. High load has a negative impact on system components, which can lead to malfunctions and data loss. To avoid this, the article discusses several design and monitoring approaches, the observance of which will help prevent system malfunctioning. The article describes the most popular way to distribute the area of responsibility of each service, in accordance with the DDD pattern, the use of which will allow you to separate the components of the system logically by use and physically when scaling the system. This approach will also be useful when scaling a team and allow developers to work independently on different system components without interfering with each other. The integration of new people into the project will also take the shortest possible time. When designing the system architecture, it is worth paying attention to the scheme of interaction between services. Using the CQRS pattern allows you to separate reading and writing into different components, which later allows the user to quickly receive a response from the system. Particular attention in the article is paid to monitoring the system, since with an increase in the size of the system, the time to search for errors in the system reaches a large amount of time, which can lead to a long unavailability of the system, which will entail the loss of clients. All the methods described in the article have been applied on many projects, for example, MTS POISK. Thanks to a properly designed system, it was possible to reduce the waiting time for a service response from two minutes to several seconds without losing the quality of the result, and a sophisticated system monitoring system allows you to monitor all processes within the system in real time and prevent accidents. As a result, at the beginning of the system design, special attention should be paid to the architecture, the issue of monitoring and testing the system. Subsequently, these temporary investments will reduce the risks of data loss and system unavailability.

1. High-load Systems. URL: https://flyoutsourcing.com/high-load-systems.html, accessed 17.06.2020.

2. Игумнов А.О., Сонькин Д.М. Об одном из подходов к оптимизации высоконагруженных систем на примере системы диспетчерского управления таксомоторным парком. Науковедение, 2013 г., №2(15), 7 стр. / Igumnov A.O., Sonkin D.M. One approache to optimize highload systems on example of dispatching taxi system. Naukovedenie, №2(15), 7 p. (in Russian).

3. Rajković P., Janković D., Milenković A. Using cqrs pattern for improving performances in medical information systems. In Proc. of the 6th Balkan Conference in Informatics, 2013, pp. 86-91.

4. Pacheco V. F. Microservice Patterns and Best Practices: Explore patterns like CQRS and event sourcing to create scalable, maintainable, and testable microservices. Packt Publishing, 2018, 366 p.

5. What are NoSQL databases? URL: https://aws.amazon.com/ru/nosql/, accessed 17.06.2020.

6. Обзор NOSQL баз данных / NOSQL Database Overview. URL: https://oracle-patches.com/db/3688-обзор-nosql-баз-данных, accessed 05.09.2020 (in Russian).

7. Redis. URL: https://redis.io/, accessed 28.10.2020.

8. ArangoDb, URL: https://www.arangodb.com/, accessed 28.10.2020.

9. What Is ClickHouse? URL: https://clickhouse.tech/docs/en/, accessed 28.10.2020.

10. MongoDb. URL: https://www.mongodb.com/, accessed 17.07.2020.

11. Postgres Professional. URL: https://habr.com/ru/company/postgrespro/blog/458186/, accessed 06.09.2020 (in Russian).

12. Design a DDD-oriented microservice. URL: https://docs.microsoft.com/ru-ru/dotnet/architecture/microservices/microservice-ddd-cqrs-patterns/ddd-oriented-microservice, accessed 06.09.2020.

13. Apache Kafka. URL: https://kafka.apache.org/, accessed 28.10.2020.

14. Pattern: Command Query Responsibility Segregation (CQRS). URL: https://microservices.io/patterns/data/cqrs.html, accessed 25.06.2020.

15. Meyer B. Object-Oriented Software Construction. Prentice Hall, 1994, 534 p.

16. МТС ПОИСК / MTS Search. URL: https://poisk.mts.ru/, accessed 25.06.2020 (in Russian).