Anomaly Detection in Computer System Logs Using semi-supervised Learning and Natural Language Processing

The detection of anomalies in computer system logs is crucial for maintaining reliable technological infrastructures. This study introduces a novel approach combining semi-supervised learning with Natural Language Processing to analyze log files for early identification of potential system failures. The methodology employs a specialized log parser based on semantic graphs alongside context-independent embedding models for text vectorization, focusing on collective rather than point anomalies. Experiments were conducted on both the public HDFS dataset and a proprietary Vertica database dataset containing over 1 113 million logs. Results demonstrate that the obtained solution based on autoencoders with convolutional layers can effectively detect system anomalies when paired with appropriate preprocessing techniques. The approach achieved impressive performance metrics on the HDFS dataset, particularly when using TF-IDF token weighting, with a Fault Detection Rate of 0.982 and ROC AUC of 0.811. Additionally, testing on the Vertica dataset successfully identified anomalous periods preceding system failures. The findings indicate that predictive maintenance approaches traditionally applied to technical equipment can be successfully adapted for computer systems, enabling proactive intervention before critical failures occur and potentially reducing the significant costs associated with system downtime.

1. Shepherd D. Why DNS exploits continue to be a top attack vector in 2024. Tahawultech, 2024. Available at: https://www.tahawultech.com/insight/why-dns-exploits-continue-to-be-a-top-attack-vector-in-2024, accessed 05.11.2025.

2. Shkodyrev V.P., Yagafarov K.I., Bashtovenko V.A., Ilyina E.E. Review of anomaly detection methods in data streams. In Proceedings of the Second Conference on Software Engineering and Information Management, St. Petersburg, Russia, 2017, vol. 1864.

3. He P., Zhu J., Zheng Z., Lyu M.R. Drain: An online log parsing approach with fixed depth tree. In IEEE International Conference on Web Services (ICWS), 2017, pp. 33-40. DOI: 10.1109/ICWS.2017.13.

4. Du M., Li F. Spell: Streaming parsing of system event logs. In 2016 IEEE 16th International Conference on Data Mining (ICDM), 2016, pp. 859-864. DOI: 10.1109/ICDM.2016.0103.

5. Bojanowski P., Grave E., Joulin A., Mikolov T. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5, 2017, 135-146. DOI: 10.1162/tacl_a_00051.

6. Sammut C., Webb G.I. TF-IDF. In Encyclopedia of Machine Learning; Springer US: Boston, MA, 2010, pp. 986-987. DOI: 10.1007/978-0-387-30164-8_832.

7. Zhang X., Xu Y., Lin Q., Qiao B., Zhang H., Dang Y., Xie C., Yang X., Cheng Q., Li Z. et al. Robust log-based anomaly detection on unstable log data. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2019, pp. 807-817. DOI: 10.1145/3338906.3338931.

8. Lu S., Wei X., Li Y., Wang L. Detecting anomaly in big data system logs using convolutional neural network. In 2018 IEEE 16th International Conference on Dependable, Autonomic and Secure Computing, 16th International Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), 2018; pp. 151-158. DOI: 10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00037.

9. Du M., Li F., Zheng G., Srikumar V. DeepLog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1285-1298. DOI: 10.1145/3133956.3134015.

10. Meng W., Liu Y., Zhu Y., Zhang S., Pei D., Liu Y., Chen Y., Zhang R., Tao S., Sun P. et al. LogAnomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI), 2019, pp. 4739-4745. DOI: 10.24963/ijcai.2019/658.

11. Guo H., Yuan S., Wu X. LogBERT: Log anomaly detection via BERT. In 2021 International Joint Conference on Neural Networks, 2021, pp. 1-8. DOI:10.48550/arXiv.2103.04475.

12. Yang L., Chen J., Wang Z., Wang W., Jiang J., Dong X., Zhang W. semi-supervised log-based anomaly detection via probabilistic label estimation. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021, pp. 1448-1460. DOI:10.1109/ICSE43902.2021.00130.

13. Han X., Yuan S., Trabelsi M. LogGPT: Log Anomaly Detection via GPT. In 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy, 2023, pp. 1117-1122. DOI: 10.1109/BigData59044.2023.10386543.

14. Guan W., Cao J., Qian S., Gao J. LogLLM: Log-based Anomaly Detection Using Large Language Models, 2024. DOI:10.48550/arXiv.2411.08561.

15. Lin Y., Deng H., Li X. FastLogAD: Log anomaly detection with mask-guided pseudo anomaly generation and discrimination. arXiv preprint, 2024. DOI: arXiv:2404.08750.

16. Nedelkoski S., Bogatinovski J., Acke A., Cardoso J., Kao O. Self-attentive classification-based anomaly detection in unstructured logs. In 2020 IEEE International Conference on Data Mining, 2020, pp. 1196-1201. DOI: 10.1109/ICDM50108.2020.00148.

17. Shchetinin E. Yu., Glushkova A. G., Sevastianov L. A. Anomaly electrocardiograms automatic detection with unsupervised deep learning methods. In Distributed Computer and Communication Networks: Control, Computation, Communications; Vishnevskiy V. M., Samouylov K. E., Kozyrev, D. V., Eds.; Springer Nature Switzerland: Cham, 2022, pp. 117-131. DOI: 10.1007/978-3-031-23207-7_10.

18. Xu W., Huang L., Fox A., Patterson D., Jordan M. I. Detecting large-scale system problems by mining console logs. In Proceedings of the 22nd Symposium on Operating Systems Principles, ACM, 2009; pp. 117-132. DOI: 10.1145/1629575.1629587.

19. Oliner A. J., Stearley J. What Supercomputers Say: A Study of Five System Logs. In Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2007.

20. Kiriachek V., Salpagarov S. Predictive diagnostics of computer systems logs using natural language processing techniques. Discrete and Continuous Models and Applied Computational Science, 2025, 33(2), 172-183. DOI: 10.22363/2658-4670-2025-33-2-172-183.

21. GE SmartSignal. Available at: https://www.ge.com/digital/applications/asset-performance-management-apm-software/smartsignal-predictive-analytics, accessed 05.11.2025.

22. Farzad A., Gulliver T. A. Unsupervised log message anomaly detection. ICT Express, 2020, pp. 229-237. DOI: 10.1016/j.icte.2020.06.003.

23. Catillo M., Pecchia A., Villano U. Autolog: Anomaly detection by deep autoencoding of system logs. Expert Systems with Applications, 191, 2022. DOI: 10.1016/j.eswa.2021.116263.

24. Schölkopf B., Platt J.C., Shawe-Taylor J., Smola A.J., Williamson R.C. Estimating the support of a high-dimensional distribution. Neural Computation, 13 (7), 2001, pp. 1443-1471. DOI: 10.1162/089976601750264965.

25. Liu F.T., Ting K.M., Zhou Z.H. Isolation forest. In 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy, 2008, pp. 413-422. DOI: 10.1109/ICDM.2008.17.

26. Breunig M., Kröger P., Ng R., Sander J. LoF: Identifying density-based local outliers. ACM Sigmod Record, 29, 2000, pp. 93-104. DOI: 10.1145/342009.335388.

27. Rousseeuw P. J., Van Driessen K. A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41(3), 1999, p. 212. DOI: 10.1080/00401706.1999.10485670.

28. Khachumov M.V., Khachumov V.M., Kovalev A.K., Panov A.I. Pattern-Recognition Tools and Their Applications. Pattern Recognit. Image Anal. 2023, 33, 28-38. DOI 10.1134/S1054661823010029.

29. Vertica: Big data analytics on-premises, in the cloud, or on hadoop. Available at: https://www.vertica.com/, accessed 03.04.2023.

30. Mikolov T., Chen K., Corrado G., Dean J. Efficient estimation of word representations in vector space, 2013. DOI:10.48550/arXiv.1301.3781.

31. Pennington J., Socher R., Manning C. D. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 2014, pp. 1532-1543. DOI: 10.3115/v1/D14-1162.

32. Devlin J., Chang M.W., Lee K., Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019. DOI: 10.18653/v1/N19-1423.

33. Zhang L., Jia T., Jia M., Li Y., Yang Y., Wu Z. Multivariate Log-based Anomaly Detection for Distributed Database, 2024. DOI: 10.48550/arXiv.2406.07976.

34. Shibzukhov Z.M. On a Robust Approach to Search for Cluster Centers. Autom Remote Control 82, 1742-1751 (2021). DOI: 10.1134/S0005117921100118.

35. Popov D., Terentev E., Serenko D., Sochenkov I., Buyanov I. Transferring Natural Language Datasets Between Languages Using Large Language Models for Modern Decision Support and Sci-Tech Analytical Systems. Big Data Cogn. Comput., 2025, 9, 116. DOI: 10.3390/bdcc9050116.