Subword-Level Grammatical Error Correction: A Universal Approach
Ildar Airatovich KHABUTDINOV,
Andrey Valerievich GRABOVOY,
Yury Viktorovich CHEKHOVICH,
Alexandr Sergeevich KILDYAKOV,
Andrey Alexandrovich IVAKHNENKO https://doi.org/10.15514/ISPRAS-2026-38(3)-11
Abstract
In this study, we propose a fully automatic methodology for data generation, correction rule vocabulary construction, and Sequence Tagging model training that specifically targets Grammatical Error Correction. Our approach operates at the SentencePiece subword level, using basic transformations – keep, append, replace and delete – that are universally applicable across languages, thereby eliminating the need for grammar-specific operations. By using the Levenshtein algorithm to generate ground truth corrections and editorial prescriptions, we obtained a completely invariant and language-independent dataset generation process. We applied our method to the Sequence Tagging model GECToR and achieved comparable quality results for English with F0.5 scores of 62.4 on the CoNLL-2014 (test set) and 61.9 on the BEA-2019 (test set), without manual rule design or manual annotation of error spans/types. The results indicate that subword-level universal edits can provide a practical alternative to grammar-specific operations, while requiring only parallel correction data.
Supporting agencies: The authors acknowledge the support of this research by the Ministry of Science and Higher Education of the Russian Federation under agreement No. 075-03-2025-662/13 (October 29, 2025), associated with the project» Applied Research on the Implementation of Artificial Intelligence Technologies in Higher Education".
About the Authors
Ildar Airatovich KHABUTDINOV
Moscow Institute of Physics and Technology
Russian Federation
Russian Federation
PhD student at the Moscow Institute of Physics and Technology (MIPT), Senior Developer in the GigaChat team at Sber. Research interests: natural language processing, large language models, neural network training and fine-tuning methods, language model alignment methods, distributed training.
Andrey Valerievich GRABOVOY
Moscow Institute of Physics and Technology, V. A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences
Russian Federation
Russian Federation
Cand. Sci. (Phys.-Math,), senior scientist of the Laboratory 42 of Data Mining at the V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences since 2025, Associate Professor at the Department of Intelligent Systems, Moscow Institute of Physics and Technology (National Research University). Research Interests: include pattern recognition, prior distributions of model parameters, model distillations, nlp.
Yury Viktorovich CHEKHOVICH
V. A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences
Russian Federation
Russian Federation
Cand. Sci (Math). Head of the Laboratory 42 of Data Mining at the V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences since 2025. Research Interest: machine learning, artificial intelligence, natural language processing, pattern recognition, high-load applications.
Alexandr Sergeevich KILDYAKOV
V. A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences
Russian Federation
Russian Federation
Scientist of the Laboratory 42 of Data Mining at the V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences since 2025. Research Interest: machine learning, artificial intelligence, natural language processing, pattern recognition, high-load applications.
Andrey Alexandrovich IVAKHNENKO
V. A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences
Russian Federation
Russian Federation
Cand. Sci. (Phys.-Math,), Scientist of the Laboratory 42 of Data Mining at the V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences since 2025. Research interests: optimization of information systems architecture, machine learning, artificial intelligence, natural language processing, pattern recognition, high-load applications..
References
1. Rozovskaya A., Roth D. Grammatical Error Correction: Machine Translation and Classifiers. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016, pp. 2205-2215.
2. Yuan Z., Stahlberg F., Rei M., Byrne B., Yannakoudakis H. Neural and FST-based approaches to grammatical error correction. Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Italy, 2019, pp. 228-239.
3. Bryant C., Ng H. How Far are We from Fully Automatic High Quality Grammatical Error Correction? Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 2015, pp. 697-707.
4. Patel H., Rajput D., Gadekallu T., Iwendi C., Bashir A., Jo H. A review on classification of imbalanced data for wireless sensor networks. International Journal of Distributed Sensor Networks, vol. 16, 2020.
5. Flickinger D., Yu J. Toward More Precision in Correction of Grammatical Errors. Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task, Sofia, Bulgaria, 2013, pp. 68-73.
6. Yuan X., Pham D., Davidson S., Yu Z. ErAConD: Error Annotated Conversational Dialog Dataset for Grammatical Error Correction. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, United States, 2022, pp. 76-84.
7. Lee J., Seneff S. An analysis of grammatical errors in non-native speech in english. 2008 IEEE Spoken Language Technology Workshop, 2008, pp. 89-92.
8. Ng H., Wu S., Briscoe T., Hadiwinoto C., Susanto R. Bryant C. Encode, The CoNLL-2014 Shared Task on Grammatical Error Correction. Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, Baltimore, Maryland, 2014, pp. 1-14.
9. Omelianchuk K., Atrasevych V., Chernodub A., Skurzhanskyi O. GECToR – Grammatical Error Correction: Tag, Not Rewrite. Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, Seattle, WA, USA, 2020, pp. 163-170.
10. Khabutdinov I., Chashchin A., Grabovoy A., Kildyakov A., Chekhovich Yu. RuGECToR: Rule-Based Neural Network Model for Russian Language Grammatical Error Correction. Program. Comput. Softw., vol. 50, issue 4, 2024, pp. 315-321.
11. Malmi E., Krause S., Rothe S., Mirylenka D., Severyn A. Encode, Tag, Realize: High-Precision Text Editing. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 2019, pp. 5054-5065.
12. Rothe S., Mallinson J., Malmi E., Krause E., Severyn A. A Simple Recipe for Multilingual Grammatical Error Correction. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021, pp. 702 707.
13. Grundkiewicz R., Junczys-Dowmunt M., Heafield K. Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data. Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Italy, 2019, pp. 252-263.
14. Rozovskaya A., Roth D. Grammar Error Correction in Morphologically Rich Languages: The Case of Russian. Transactions of the Association for Computational Linguistics, vol. 7, 2019, pp. 1--17.
15. Kudo T., Richardson J. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Brussels, Belgium, 2018, pp. 66-71.
16. Limisiewicz T., Balhar J., Marecek D. Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages. Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, 2023, pp. 5661-5681.
17. Asvarov A., Grabovoy A. The Impact of Multilinguality and Tokenization on Statistical Machine Translation. 35th Conference of Open Innovations Association (FRUCT), Tampere, Finland, 2024, pp. 149-157.
18. Liu Y., Ott M., et. al. RoBERTa: A Robustly Optimized BERT Pretraining Approach, 2019. Cite arxiv:1907.11692.
19. Devlin J., Chang M., Lee K., Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, 2019, pp. 4171-4186.
20. Yang Z., Dai Z., Yang Y., Carbonell J., Salakhutdinov R., Le Q. XLNet: generalized autoregressive pretraining for language understanding. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2019.
21. Dahlmeier D., Ng H. Better Evaluation for Grammatical Error Correction. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, Canada, 2012, pp. 568-572.
22. Awasthi A., Sarawagi S., et. al. Parallel Iterative Edit Models for Local Sequence Transduction. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 2019, pp. 4259-4269.
23. Mizumoto T., Komachi M., Nagata M., Matsumoto J. Mining revision log of language learning SNS for automated Japanese error correction of second language learners. Proc. of 5th International Joint Conference on Natural Language Processing, 2011, pp. 147-155.
24. Dahlmeier D., Ng H., Wu S. Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English. Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, Atlanta, Georgia, 2013, pp. 22-31.
25. Yannakoudakis H., Briscoe T., Medlock B. A New Dataset and Method for Automatically Grading ESOL Texts. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, 2011, pp. 180-189.
26. Bryant C., Felice M., et. al. The BEA-2019 Shared Task on Grammatical Error Correction. Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Italy, 2019, pp. 52-75.
27. Bryant C., Felice M., Briscoe T. Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,Vancouver, Canada, 2017, pp. 793-805.
Review
For citations:
KHABUTDINOV I.A., GRABOVOY A.V., CHEKHOVICH Yu.V., KILDYAKOV A.S., IVAKHNENKO A.A. Subword-Level Grammatical Error Correction: A Universal Approach. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2026;38(3):187-196. https://doi.org/10.15514/ISPRAS-2026-38(3)-11
Email this article 