Towards a Test Automation Framework for Data Race Testing

In this paper, we examine shared memory concurrent programs and errors occurring in them, specifically data races. We design a test automation framework to develop data race revealing testing scenarios by analogy to test automation frameworks for sequential programs. The main problem complicating development of testing scenarios is the nondeterministic nature of multithreaded program executions. To provide repeatable testing scenarios we define a notion of synchronization points in the source code of the computer program where a test regulates execution of parallel threads with synchronization actions. Tests being developed with our test automation framework can be executed automatically and can be used for regression testing.

1. Beust C., Suleiman H. Next generation Java testing: TestNG and advanced concepts. – Pearson Education, 2007.

2. Massol V. JUnit in action. – 2004.

3. Hyuk Myeong. Googletest In Practice: Unit Testing Guide for C++ Programmers. – 2021.

4. Leblanc. Debugging parallel programs with instant replay //IEEE Transactions on Computers. – 1987. – Т. 100. – №. 4. – С. 471-482.

5. Wang Y. et al. Drdebug: Deterministic replay based cyclic debugging with dynamic slicing //Proceedings of annual IEEE/ACM international symposium on code generation and optimization. – 2014. – С. 98-108.

6. Thane H., Hansson H. Using deterministic replay for debugging of distributed real-time systems //Proceedings 12th Euromicro Conference on Real-Time Systems. Euromicro RTS 2000. – IEEE, 2000. – С. 265-272.

7. Serebryany K., Iskhodzhanov T. ThreadSanitizer: data race detection in practice //Proceedings of the workshop on binary instrumentation and applications. – 2009. – С. 62-71.

8. Savage S. et al. Eraser: A dynamic data race detector for multithreaded programs //ACM Transactions on Computer Systems (TOCS). – 1997. – Т. 15. – №. 4. – С. 391-411.

9. Erickson J. et al. Effective {Data-Race} Detection for the Kernel //9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10). – 2010.

10. The Kernel Concurrency Sanitizer. The Linux Kernel Organization. Available at: https://docs.kernel.org/dev-tools/kcsan.html, accessed 04.01.2025.

11. Adve S. V. et al. Detecting data races on weak memory systems //ACM SIGARCH Computer Architecture News. – 1991. – Т. 19. – №. 3. – С. 234-243.

12. Lamport L. Time, clocks, and the ordering of events in a distributed system //Concurrency: the Works of Leslie Lamport. – 2019. – С. 179-196.

13. RTCA DO-178C, Software Considerations in Airborne Systems and Equipment Certification.

14. ISO International Standard ISO/IEC 14882:2020(E) – Programming Language C++.

15. Исходный код инструмента Clade для перехвата команд сборки. Веб-ссылка: https://github.com/17451k/clade, доступно 01.04.2024.

16. Документация компилятора Clang с главой, посвящённой библиотеке libclang. Веб-ссылка: https://clang.llvm.org/doxygen/index.html, доступно 01.04.2024.

17. Yu T., Srisa-an W., Rothermel G. SimRT: An automated framework to support regression testing for data races //Proceedings of the 36th international conference on software engineering. – 2014. – С. 48-59.

18. Halpert R. L., Pickett C. J. F., Verbrugge C. Component-based lock allocation //16th International Conference on Parallel Architecture and Compilation Techniques (PACT 2007). – IEEE, 2007. – С. 353 - 364.

19. Huang J., Liu P., Zhang C. LEAP: Lightweight deterministic multi-processor replay of concurrent Java programs //Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering. – 2010. – С. 207-216.

20. Musuvathi M. et al. Finding and Reproducing Heisenbugs in Concurrent Programs //OSDI. – 2008. – Т. 8. – №. 2008.

21. Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs //IEEE transactions on computers. – 1979. – Т. 100. – №. 9. – С. 690-691.

22. Said, M., Wang, C., Yang, Z., & Sakallah, K. (2011, April). Generating data race witnesses by an SMT-based analysis. In NASA Formal Methods Symposium (pp. 313-327). Springer, Berlin, Heidelberg.

23. De Moura L., Bjørner N. Satisfiability modulo theories: introduction and applications //Communications of the ACM. – 2011. – Т. 54. – №. 9. – С. 69-77.

24. Dutertre B., De Moura L. A fast linear-arithmetic solver for DPLL (T) //International Conference on Computer Aided Verification. – Berlin, Heidelberg : Springer Berlin Heidelberg, 2006. – С. 81-94.

25. https://www.rapitasystems.com/products/rapitest.