Automated testing of a TCG frontend for Qemu
https://doi.org/10.15514/ISPRAS-2019-31(5)-1
Abstract
Implementing a new target architecture in Qemu involves creation of a dynamic binary translator TCG front-end for that architecture. Testing is necessary to verify correctness of that translator component. Currently, existing TCG front-end testing systems use an approach based on a comparison with an oracle. Such oracle have the same processor architecture. And an oracle may be a real processor, a high-fidelity emulator or another binary translator. Unfortunately, such oracles are not always available. This paper is devoted to testing a target architecture implementation in Qemu when the necessary oracle is not available. The main idea is following. There is observation, a program written in a high-level programming language is expected to execute equally regardless of processor architecture. In other words, one can use a real processor with a different architecture for comparison. In this paper, it is the processor of a developer AMD64 machine. The comparison objects are the term of a high-level programming language. I.e. tests are written in C. C language was chosen for this purpose, because, on the one hand, it is fairly close to the hardware, and, on the other, it has compilers for many processor architectures. The approach is implemented in CPU Testing Tool (c2t) which is part of QDT. Source code is available at https://github.com/ispras/qdt. The tool is implemented in Python programming language and supports testing of Qemu in both full system and user level emulation modes. c2t is suitable for testing TCG front-ends which are generated by the automatic TCG front-end generation system or implemented in the classical way (manually).
Supporting agencies: This work was supported by RFBR grant № 16-29-09632
About the Authors
Dmitry Sergeevich Koltunov
Ivannikov Institute for System Programming of the Russian Academy of Sciences
Russian Federation
Research trainee
Russian Federation
Research trainee
Vasily Yuryevich Efimov
Ivannikov Institute for System Programming of the Russian Academy of Sciences
Russian Federation
Junior researcher
Russian Federation
Junior researcher
Vartan Andronikovich Padaryan
Ivannikov Institute for System Programming of the Russian Academy of Sciences;
Lomonosov Moscow State University
Russian Federation
Candidate of Physical and Mathematical Sciences, Leading Researcher at ISP RAS, Associate Professor of the System Programming Department of the the faculty of Computational Mathematics and Cybernetics of Lomonosov Moscow State University.
Russian Federation
Candidate of Physical and Mathematical Sciences, Leading Researcher at ISP RAS, Associate Professor of the System Programming Department of the the faculty of Computational Mathematics and Cybernetics of Lomonosov Moscow State University.
References
1. Ефимов В.Ю., Беззубиков А.А., Богомолов Д.А., Горемыкин О.В., Падарян В.А. Автоматизация разработки моделей устройств и вычислительных машин для QEMU. Труды ИСП РАН, том 29, вып. 6, 2017 г., стр. 77-104 / Efimov V.Yu., Bezzubikov A.A., Bogomolov D.A., Goremykin O.V., Padaryan V.A. Automation of device and machine development for QEMU. Trudy ISP RAN/Proc. ISP RAS, vol. 29, issue 6, 2017, pp. 77-104 (In Russian). DOI: 10.15514/ISPRAS-2017-29(6)-4.
2. Bezzubikov A., Belov N., Batuzov K. Automatic dynamic binary translator generation from instruction set description. In Proc. of the 2017 Ivannikov ISPRAS Open Conference, 2017, pp. 27-33. DOI: 10.1109/ISPRAS.2017.00012.
3. W.E. Howden, Theoretical and empirical studies of program testing. In Proc. of the 3rd international conference on Software engineering, 1978, pp. 305-311.
4. Lorenzo Martignoni, Roberto Paleari, Giampaolo Fresi Roglia, Danilo Bruschi. Testing CPU emulators. In Proc. of the 18th international symposium on Software testing and analysis, 2009, pp. 261-272.
5. Lorenzo Martignoni, Roberto Paleari, Giampaolo Fresi Roglia, Danilo Bruschi. Testing system virtual machines. In Proc. of the 19th international symposium on Software testing and analysis, 2010, pp. 171-182.
6. Qiuchen Yan and Stephen McCamant. Fast PokeEMU: Scaling Generated Instruction Tests Using Aggregation and State Chaining. In Proc. of the 14th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, 2018, 13 p.
7. Risu: random instruction sequence tester for userspace [online] Available at: https://git.linaro.org/people/pmaydell/risu.git/about/, accessed: 09.08.2019.
8. A.S. Kamkin, T.I. Sergeeva, S.A. Smolov, A.D. Tatarnikov, M.M. Chupilko. Extensible environment for test program generation for microprocessors. Programming and Computer Software, vol. 40, issue 1, 2014, pp 1-9.
9. Soomin Kim, Markus Faerevaag, Minkyu Jung, SeungIl Jung, DongYeop Oh, JongHyup Lee, Sang Kil Cha. Testing intermediate representations for binary analysis. In Proc. of the 32nd IEEE/ACM International Conference on Automated Software Engineering, 2017, pp. 353-364.
10. L. Martignoni, S. McCamant, P. Poosankam, D. Song, and P. Maniatis. Path-exploration lifting: Hi-fi tests for lo-fi emulators. In Proc. of the International Conference on Architectural Support for Programming Languages and Operating Systems, 2012, pp. 337-348.
11. Hao Shi, Abdulla Alwabel, and Jelena Mirkovic. Cardinal pill testing of system virtual machines. In Proceedings of the 23rd USENIX Security Symposium, 2014, pp. 271-285.
12. pyrsp. Available at: https://github.com/stef/pyrsp, accessed: 02.08.2019.
13. pyelftools. Available at: https://github.com/eliben/pyelftools, accessed:23.09.2018.
14. Qemu MSP430. Available at: https://github.com/draperlaboratory/qemu-msp, accessed: 16.07.2019.
15. ARM and Thumb-2 Instruction Set Quick Reference Card. Available at: http://infocenter.arm.com/help/topic/com.arm.doc.qrc0001m/QRC0001_UAL.pdf, accessed: 16.07.2019.
16. MIPS Instruction Reference. Available at: https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MD00565-2B-MIPS32-Q RC-01.01.pdf, accessed: 16.07.2019.
17. MSP430x2xx Family User’s Guide. Available at: http://www.ti.com/lit/ug/slau144j/slau144j.pdf, accessed: 16.07.2019.
Review
For citations:
Koltunov D.S., Efimov V.Yu., Padaryan V.A. Automated testing of a TCG frontend for Qemu. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2019;31(5):7-24. (In Russ.) https://doi.org/10.15514/ISPRAS-2019-31(5)-1
Email this article 