Machine Learning Based Congestion Control Methods: a Survey
Ivan Alexandrovich STEPANOV,
Maxim Vladimirovich POPOV,
Aleksandr Igorevich GETMAN,
Maria Kirillovna IKONNIKOVA,
Andrey Andreevich BELEVANTSEV https://doi.org/10.15514/ISPRAS-2025-37(3)-18
Abstract
Congestion control is a key aspect of modern networks. The first congestion control algorithms, such as TCP Tahoe and TCP Reno, were developed in the late 20th century, and their core concepts remain relevant to this day. With the development of high-speed networks, specialized algorithms such as TCP BIC and TCP CUBIC were created, which are adapted to these conditions. However, classical algorithms with predefined rules are not always effective in all network environments, and with the rise of 4G, 5G, and satellite communications, the congestion control issue has become increasingly relevant. This has led to the emergence of numerous works on machine learning-based congestion control algorithms, particularly reinforcement learning, which can adapt to dynamically changing network conditions. This paper presents and reviews both classical congestion control algorithms and the most popular and recent machine learning-based algorithms, along with some implementations using multipath. Additionally, it highlights the most significant challenges of machine learning-based algorithms and discusses potential directions for future research in this field.
About the Authors
Ivan Alexandrovich STEPANOV
Ivannikov Institute for System Programming of the Russian Academy of Sciences, Moscow Institute of Physics and Technology (National Research University)
Russian Federation
Russian Federation
Postgraduate student of the ISP RAS, an assistant at the Department of Computer Science and Computational Mathematics at MIPT. Research interests: network traffic analysis using machine learning.
Maxim Vladimirovich POPOV
Ivannikov Institute for System Programming of the Russian Academy of Sciences, Lomonosov Moscow State University
Russian Federation
Russian Federation
Graduate student at the Faculty of Computational Mathematics and Cybernetics of Moscow State University. Research interests: network traffic analysis, congestion control algorithms.
Aleksandr Igorevich GETMAN
Ivannikov Institute for System Programming of the Russian Academy of Sciences, Moscow Institute of Physics and Technology (National Research University), National Research University «Higher School of Economics», Lomonosov Moscow State University
Russian Federation
Russian Federation
Cand. Sci. (Phys.-Math.), senior researcher at ISP RAS, assistant at CMC MSU and MIPT, associate professor at HSE. Research interests: binary code analysis, data format recovery, network traffic analysis and classification.
Maria Kirillovna IKONNIKOVA
Ivannikov Institute for System Programming of the Russian Academy of Sciences, Moscow Institute of Physics and Technology (National Research University)
Russian Federation
Russian Federation
A junior researcher at the ISP RAS. Research interests: network traffic analysis, machine learning.
Andrey Andreevich BELEVANTSEV
Ivannikov Institute for System Programming of the Russian Academy of Sciences, Lomonosov Moscow State University
Russian Federation
Russian Federation
Dr. Sci. (Phys.-Math.), Prof., leading researcher at ISP RAS, Professor at MSU. Research interests: static analysis, program optimization, parallel programming.
References
1. Jiang H. et al. When machine learning meets congestion control: A survey and comparison //Computer Networks. – 2021. – Т. 192. – С. 108033.
2. Wei W., Gu H., Li B. Congestion control: A renaissance with machine learning //IEEE network. – 2021. – Т. 35. – №. 4. – С. 262-269.
3. Jacobson V. Congestion avoidance and control //ACM SIGCOMM computer communication review. – 1988. – Т. 18. – №. 4. – С. 314-329.
4. Mo J. et al. Analysis and comparison of TCP Reno and Vegas //IEEE INFOCOM'99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No. 99CH36320). – IEEE, 1999. – Т. 3. – С. 1556-1563.
5. Floyd S., Henderson T. The NewReno modification to TCP's fast recovery algorithm. – 1999. – №. rfc2582.
6. Kelly T. Scalable TCP: Improving performance in highspeed wide area networks //ACM SIGCOMM computer communication Review. – 2003. – Т. 33. – №. 2. – С. 83-91.
7. Floyd S. HighSpeed TCP for large congestion windows. – 2003. – №. rfc3649.
8. Xu L., Harfoush K., Rhee I. Binary increase congestion control (BIC) for fast long-distance networks //IEEE INFOCOM 2004. – IEEE, 2004. – Т. 4. – С. 2514-2524.
9. Ha S., Rhee I., Xu L. CUBIC: a new TCP-friendly high-speed TCP variant //ACM SIGOPS operating systems review. – 2008. – Т. 42. – №. 5. – С. 64-74.
10. Brakmo L. S., O'malley S. W., Peterson L. L. TCP Vegas: New techniques for congestion detection and avoidance //Proceedings of the conference on Communications architectures, protocols and applications. – 1994. – С. 24-35.
11. Srijith K. N., Jacob L., Ananda A. L. TCP Vegas-A: Solving the fairness and rerouting issues of TCP Vegas //Conference Proceedings of the 2003 IEEE International Performance, Computing, and Communications Conference, 2003. – IEEE, 2003. – С. 309-316.
12. Srijith K. N., Jacob L., Ananda A. L. TCP Vegas-A: Improving the performance of TCP Vegas //Computer communications. – 2005. – Т. 28. – №. 4. – С. 429-440.
13. Fu C. P., Liew S. C. TCP Veno: TCP enhancement for transmission over wireless access networks //IEEE Journal on selected areas in communications. – 2003. – Т. 21. – №. 2. – С. 216-228.
14. Mascolo S. et al. TCP Westwood: Bandwidth estimation for enhanced transport over wireless links //Proceedings of the 7th annual international conference on Mobile computing and networking. – 2001. – С. 287-297.
15. Cardwell N. et al. Bbr: Congestion-based congestion control: Measuring bottleneck bandwidth and round-trip propagation time //Queue. – 2016. – Т. 14. – №. 5. – С. 20-53.
16. Cardwell N. et al. BBRv2: A model-based congestion control performance optimization //Proc. IETF 106th Meeting. – 2019. – С. 1-32.
17. Zhang Y., Cui L., Tso F. P. Modest BBR: Enabling better fairness for BBR congestion control //2018 IEEE symposium on computers and communications (ISCC). – IEEE, 2018. – С. 00646-00651.
18. Arun V., Balakrishnan H. Copa: Practical {Delay-Based} congestion control for the internet //15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). – 2018. – С. 329- 342.
19. Watkins C. J. C. H., Dayan P. Q-learning //Machine learning. – 1992. – Т. 8. – С. 279-292
20. Gu S. et al. Continuous deep q-learning with model-based acceleration //International conference on machine learning. – PMLR, 2016. – С. 2829-2838.
21. Konda V., Tsitsiklis J. Actor-critic algorithms //Advances in neural information processing systems. – 1999. – Т. 12.
22. Labao A. B., Martija M. A. M., Naval P. C. A3C-GS: Adaptive moment gradient sharing with locks for asynchronous actor–critic agents //IEEE Transactions on Neural Networks and Learning Systems. – 2020. – Т. 32. – №. 3. – С. 1162-1176.
23. Schulman J. et al. Proximal policy optimization algorithms //arXiv preprint arXiv:1707.06347. – 2017.
24. El Khayat I., Geurts P., Leduc G. Improving TCP in wireless networks with an adaptive machine-learnt classifier of packet loss causes //NETWORKING 2005. Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communications Systems: 4th International IFIP-TC6 Networking Conference, Waterloo, Canada, May 2-6, 2005. Proceedings 4. – Springer Berlin Heidelberg, 2005. – С. 549-560.
25. Geurts P., El Khayat I., Leduc G. A machine learning approach to improve congestion control over wireless computer networks //Fourth IEEE International Conference on Data Mining (ICDM'04). – IEEE, 2004. – С. 383-386.
26. Jayaraj A., Venkatesh T., Murthy C. S. R. Loss classification in optical burst switching networks using machine learning techniques: improving the performance of tcp //IEEE Journal on Selected Areas in Communications. – 2008. – Т. 26. – №. 6. – С. 45-54.
27. Arouche Nunes B. A. et al. A machine learning framework for TCP round-trip time estimation //EURASIP Journal on Wireless Communications and Networking. – 2014. – Т. 2014. – С. 1-22.
28. Dai T., Zhang X., Guo Z. Learning-based congestion control for internet video communication over wireless networks //2018 IEEE International Symposium on Circuits and Systems (ISCAS). – IEEE, 2018. – С. 1-5.
29. Parlos A. G. Identification of the internet end-to-end delay dynamics using multi-step neuro-predictors //Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No. 02CH37290). – IEEE, 2002. – Т. 3. – С. 2460-2465.
30. Bui V. et al. Long horizon end-to-end delay forecasts: A multi-step-ahead hybrid approach //2007 12th IEEE Symposium on Computers and Communications. – IEEE, 2007. – С. 825-832.
31. Belhaj S., Tagina M. Modeling and Prediction of the Internet End-to-end Delay using Recurrent Neural Networks //J. Networks. – 2009. – Т. 4. – №. 6. – С. 528-535.
32. Ahmed A. H. et al. Predicting high delays in mobile broadband networks //IEEE Access. – 2021. – Т. 9. – С. 168999-169013.
33. Li W. et al. QTCP: Adaptive congestion control with reinforcement learning //IEEE Transactions on Network Science and Engineering. – 2018. – Т. 6. – №. 3. – С. 445-458.
34. Jay N. et al. A deep reinforcement learning perspective on internet congestion control //International Conference on Machine Learning. – PMLR, 2019. – С. 3050-3059.
35. Fang J. et al. Reinforcement learning for bandwidth estimation and congestion control in real-time communications //arXiv preprint arXiv:1912.02222. – 2019.
36. Abbasloo S., Yen C. Y., Chao H. J. Classic meets modern: A pragmatic learning-based congestion control for the internet //Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. – 2020. – С. 632-647.
37. Sivakumar V. et al. Mvfst-rl: An asynchronous rl framework for congestion control with delayed actions //arXiv preprint arXiv:1910.04054. – 2019.
38. Zhang Z. et al. PBQ-Enhanced QUIC: QUIC with Deep Reinforcement Learning Congestion Control Mechanism //Entropy. – 2023. – Т. 25. – №. 2. – С. 294.
39. Tian H. et al. Spine: An efficient DRL-based congestion control with ultra-low overhead //Proceedings of the 18th International Conference on emerging Networking Experiments and Technologies. – 2022. – С. 261-275.
40. Shi H., Wang J. Intelligent TCP congestion control policy optimization //Applied Sciences. – 2023. – Т. 13. – №. 11. – С. 6644.
41. Andrade-Zambrano A. R. et al. A Reinforcement Learning Congestion Control Algorithm for Smart Grid Networks //IEEE Access. – 2024.
42. Wischik D. et al. Design, implementation and evaluation of congestion control for multipath {TCP} //8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11). – 2011.
43. Nguyen K. et al. An approach to reinforce multipath TCP with path-aware information //Sensors. – 2019. – Т. 19. – №. 3. – С. 476.
44. Chao L. et al. A brief review of multipath tcp for vehicular networks //Sensors. – 2021. – Т. 21. – №. 8. – С. 2793.
45. Deng S. et al. WiFi, LTE, or both? Measuring multi-homed wireless internet performance //Proceedings of the 2014 Conference on Internet Measurement Conference. – 2014. – С. 181-194.
46. Scharf M., Kiesel S. NXG03-5: Head-of-line Blocking in TCP and SCTP: Analysis and Measurements //IEEE Globecom 2006. – IEEE, 2006. – С. 1-5.
47. De Coninck Q., Bonaventure O. Multipath quic: Design and evaluation //Proceedings of the 13th international conference on emerging networking experiments and technologies. – 2017. – С. 160-166.
48. Viernickel T. et al. Multipath QUIC: A deployable multipath transport protocol //2018 IEEE International Conference on Communications (ICC). – IEEE, 2018. – С. 1-7.
49. Wang S. et al. Deep reinforcement learning for dynamic multichannel access //International Conference on Computing, Networking and Communications (ICNC). – 2017. – С. 257-265.
50. Lee S., Yoo J. Reinforcement learning based multipath QUIC scheduler for multimedia streaming //Sensors. – 2022. – Т. 22. – №. 17. – С. 6333.
51. Luo J., Su X., Liu B. A reinforcement learning approach for multipath TCP data scheduling //2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC). – IEEE, 2019. – С. 0276-0280.
52. Roselló M. M. Multi-path scheduling with deep reinforcement learning //2019 European Conference on Networks and Communications (EuCNC). – IEEE, 2019. – С. 400-405.
53. Yu C. et al. Reliable cybertwin-driven concurrent multipath transfer with deep reinforcement learning //IEEE Internet of Things Journal. – 2021. – Т. 8. – №. 22. – С. 16207-16218.
54. Deutschmann J., Hielscher K. S. J., German R. An ns-3 model for multipath communication with terrestrial and satellite links //Measurement, Modelling and Evaluation of Computing Systems: 20th International GI/ITG Conference, MMB 2020, Saarbrücken, Germany, March 16–18, 2020, Proceedings 20. – Springer International Publishing, 2020. – С. 65-81.
55. Mai T. et al. Self-learning congestion control of MPTCP in satellites communications //2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC). – IEEE, 2019. – С. 775-780.
56. Xing Z. et al. A multipath routing algorithm for satellite networks based on service demand and traffic awareness //Frontiers of Information Technology & Electronic Engineering. – 2023. – Т. 24. – №. 6. – С. 844-858.
57. Lv G. et al. Chorus: Coordinating Mobile Multipath Scheduling and Adaptive Video Streaming //Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. – 2024. – С. 246-262.
58. Berenji H. R. Fuzzy Q-learning: a new approach for fuzzy dynamic programming //Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference. – IEEE, 1994. – С. 486-491.
59. Zhou W. et al. TCP Vegas-V: Improving the performance of TCP Vegas //International Conference on Automatic Control and Artificial Intelligence (ACAI 2012). – IET, 2012. – С. 2034-2039.
60. Dong M. et al. {PCC} vivace: {Online-Learning} congestion control //15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). – 2018. – С. 343-356.
61. Dong M. et al. {PCC}: Re-architecting congestion control for consistent high performance //12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). – 2015. – С. 395-408.
62. De Oliveira R. L. S. et al. Using mininet for emulation and prototyping software-defined networks //2014 IEEE Colombian conference on communications and computing (COLCOM). – Ieee, 2014. – С. 1-6.
63. Riley G. F., Henderson T. R. The ns-3 network simulator //Modeling and tools for network simulation. – Berlin, Heidelberg: Springer Berlin Heidelberg, 2010. – С. 15-34.
64. Beale J., Orebaugh A., Ramirez G. Wireshark & Ethereal network protocol analyzer toolkit. – Elsevier, 2006.
65. Deri L. et al. ndpi: Open-source high-speed deep packet inspection //2014 International Wireless Communications and Mobile Computing Conference (IWCMC). – IEEE, 2014. – С. 617-622.
66. Mishra A., Leong B. Containing the Cambrian Explosion in QUIC Congestion Control //Proceedings of the 2023 ACM on Internet Measurement Conference. – 2023. – С. 526-539.
Review
For citations:
STEPANOV I.A., POPOV M.V., GETMAN A.I., IKONNIKOVA M.K., BELEVANTSEV A.A. Machine Learning Based Congestion Control Methods: a Survey. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2025;37(3):251-276. (In Russ.) https://doi.org/10.15514/ISPRAS-2025-37(3)-18
Email this article 