Evolution of UniTESK Test Development Technology
https://doi.org/10.15514/ISPRAS-2014-26(1)-1
Abstract
The paper presents the basic principles of UniTESK technology intended for test construction based on formal models. It also summarizes experience of using UniTESK in large test development projects for software and hardware systems, including telecommunication protocols, basic and standard interfaces of operating systems, microprocessor units. The paper provides a vision of the future technology development for higher scalability of test suites constructed
About the Authors
V. Kuliamin
Institute for System Programming of RAS
Russian Federation
Russian Federation
A. Petrenko
Institute for System Programming of RAS
Russian Federation
Russian Federation
References
1. I. Bourdonov, A. Kossatchev, V. Kuliamin, A. Petrenko. UniTesK Test Suite Architecture. Proceedings of FME’2002, Kopenhagen, Denmark, LNCS 2391:77-88, Springer-Verlag, 2002.
2. V. V. Kuliamin, A. K. Petrenko, A. S. Kossatchev, and I. B. Burdonov. The UniTesK Approach to Designing Test Suites. Programming and Computer Software, 29(6):310-322, 2003.
3. ISO/IEC TR 19759 Software Engineering — Guide to the Software Engineering Body of Knowledge (SWEBOK). Geneva, Switzerland: ISO, 2005.
4. L. Baresi, M. Young. Test Oracles. Tech. Report CIS-TR-01-02. 2001, http://www.cs.uoregon.edu/~michal/pubs/oracles.html.
5. V. V. Kuliamin. Organizatsiya slozhnykh testovykh naborov [Organization of complex test suites]. Trudy ISP RАN [Proceedings of ISP RAS], 17:9-24, 2009 (in Russian).
6. B. Meyer. Applying Design by Contract. IEEE Computer,25(10): 40-51, October 1992.
7. I. Bourdonov, A. Kossatchev, A. Petrenko, D. Galter. KVEST: Automated Generation of Test Suites from Formal Specifications. Proceedings of FM’99, Toulouse, France, LNCS 1708:608-621, Springer-Verlag, 1999.
8.
9.
10. I. B. Bourdonov, A. V. Demakov, A. A. Jarov, A. S. Kossatchev, V. V. Kuliamin, A. K. Petrenko, S. V. Zelenov. Java Specification Extension for Automated Test Development. Proceedings of PSI’2001, Novosibirsk, Russia, LNCS 2244:301-307, Springer-Verlag, 2001.
11. G. V. Kluchnikov, A. S. Kossatchev, N. V. Pakulin, A. K. Petrenko, V. Z. Shnitman. Primenenie formal'nykh metodov dlya testirovaniya realizatsii IPv6 [Using formal methods for IPv6 implementation testing]. Trudy ISP RАN [Proceedings of ISP RAS], 4:121-140, 2003 (in Russian).
12. V. V. Kuliamin. Component architecture of model-based testing environment. Programming and Computer Software, 36(5):289-305, 2010.
13. V. V. Kuliamin, A. K. Petrenko, N. V. Pakoulin, A. S. Kossatchev, I. B. Bourdonov. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proceedings of PSI’2003, Novosibirsk, Russia, LNCS 2890:450-461, Springer-Verlag, 2003.
14. V. Kuliamin, A. Petrenko, N. Pakoulin. Practical Approach to Specification and Conformance Testing of Distributed Network Applications. Proceedings of ISAS'2005, Berlin, Germany, LNCS 3694:68-83, Springer-Verlag, 2005.
15. A. V. Khoroshilov. Spetsifikatsiya i testirovanie komponentov s asinkhronnym interfejsom [Specification and testing of components with asynchronous interface]. PhD Thesis, ISP RAS, Moscow, 2006 (in Russian).
16. I. B. Burdonov, A. S. Kossatchev, and V. V. Kulyamin. Application of Finite Automatons for Program Testing. Programming and Computer Software, 26(2):61-73, 2000.
17. I. B. Burdonov, A. S. Kossatchev, and V. V. Kuliamin. Irredundant Algorithms for Traversing Directed Graphs: The Deterministic Case. Programming and Computer Software, 29(5):245-258, 2003.
18. H. Zhu, P. A. V. Hall, J. H. R. May. Software Unit Test Coverage and Adequacy. ACM Computing Surveys, 29(4):366-427, Dec. 1997.
19. A. Grinevich, A. Khoroshilov, V. Kuliamin, D. Markovtsev, A. Petrenko, V. Rubanov. Formal Methods in Industrial Software Standards Enforcement. Proceedings of PSI’2006, Novosibirsk, Russia, LNCS 4378:459-469, 2006.
20. OLVER project, http://linuxtesting.org
21. A. Maksimov. Requirements-Based Conformance Testing of ARINC 653 Real-Time Operating Systems. Proceedings of Data Systems In Aerospace (DASIA) 2010, ESA SP-682, ISBN 978-92-9221-246-9, 2010.
22. G. V. Kluchnikov, A. S. Kossatchev, N. V. Pakulin, A. K. Petrenko, V. Z. Shnitman. Primenenie formal'nykh metodov dlya testirovaniya Mobile IPv6 [Using formal methods for Mobile IPv6 conformance testing]. 2-nd International Conference “Next Generation Internet”, pp. 20-25, Yaroslavl, Russia, 2003 (in Russian).
23. V. P. Ivannikov, A. S. Kamkin, A. S. Kossatchev, V. V. Kuliamin, and A. K. Petrenko. The use of contract specifications for representing requirements and for functional testing of hardware models. Programming and Computer Software, 33(5):272-282, 2007.
24. W. Grieskamp. Microsoft’s Protocol Documentation Program: A Success Story for Model-Based Testing. Testing – Practice and Research Techniques. Lecture Notes in Computer Science, vol. 6303, p. 7, Springer, 2010.
25. P. Godefroid, N. Klarlund, K. Sen. DART: Directed Automated Random Testing. ACM SIGPLAN Notices — Proceedings of PLDI 2005, 40(6):213-223, 2005.
26. K. Sen, D. Marinov, G. Agha. CUTE: a concolic unit testing engine for C. Proceedings of ESES/FSE, pp. 263–272, 2005.
27. C. Cadar, V. Ganesh, P. Pawloski, D. Dill, D. Engler. EXE: Automatically Generating Inputs of Death. Proceedings of the 13-th International Conference on Computer and Communications Security CCS 2006, pp. 322-335.
28. C. Pacheco, S. K. Lahiri, M. D. Ernst, T. Ball. Feedback-Directed Random Test Generation. Proc. of International Conference on Software Engineering, pp. 75-84, 2007.
29. I. B. Burdonov, S. G. Groshev, A. V. Demakov, A. S. Kamkin, A. S. Kossatchev, A. A. Sortov. Parallel'noe testirovanie bol'shikh avtomatnykh modelej [Parallel testing of large FSM models]. Vestnik NNGU [Bulletin of NNSU], № 3, 2011, pp. 187-193 (in Russian).
30. I. B. Burdonov, A. S. Kossatchev. Obkhod neizvestnogo grafa kollektivom avtomatov [Exploration of unknown graph by a set of autonata]. Proc. of International Conference "Science Service in the Internet: all facets of parallelism ". 2013, pp. 228-232 (in Russian).
31. C. Flanagan, P. Godefroid. Dynamic Partial-Order Reduction for Model Checking Software. ACM SIGPLAN Notices— Proceedings of POPL 2005, 40(1):110-121, 2005.
32. Y. Yang, X. Chen, G. Gopalakrishnan, R. Kirby. Efficient Stateful Dynamic Partial Order Reduction. Proceedings of SPIN 2008, LNCS 5156:288-305, Springer, 2008.
33. V. P. Ivannikov, A. K. Petrenko, V. V. Kuliamin, A. V. Maksimov. Opyt ispol'zovaniya UniTESK kak zerkalo razvitiya tekhnologij testirovaniya na osnove modelej [Experiences of UniTESK applications reflecting model based testing technology development]. Trudy ISP RАN [Proceedings of ISP RAS], 23:207-218, 2013. DOI: 10.15514/ISPRAS-2013-24-11. (in Russian).
34. I. B. Burdonov, A. S. Kossatchev. Razvitie teorii konformnosti: semantiki, formal'nye modeli, algoritmy [Development of conformance testing theory: models, semantics, algorithms]. Trudy ISP RАN [Proceedings of ISP RAS], 2014. DOI: 10.15514/ISPRAS-2014-26(1)-2. (in Russian).
35. E. A. Gerlits, A. V. Maksimov, A. K. Petrenko, A. V. Khoroshilov. Testirovanie operatsionnykh sistem [Testing of operating systems]. Trudy ISP RАN [Proceedings of ISP RAS], 2014. DOI: 10.15514/ISPRAS-2014-26(1)-3. (in Russian).
36. N. V. Pakulin, V. Z. Shnitman. Аvtomatizatsiya testirovaniya sootvetstviya dlya telekommunikatsionnykh protokolov [Protocol conformance testing automation]. Trudy ISP RАN [Proceedings of ISP RAS], 2014. DOI: 10.15514/ISPRAS-2014-26(1)-4. (in Russian).
37. A. S. Kamkin, A. M. Kotsyniak, S. A. Smolov, A. D. Tatarnikov, M. M. Chupilko. Sredstva funktsional'noj verifikatsii mikroprotsessorov [Microprocessor functional verification tools and methods]. Trudy ISP RАN [Proceedings of ISP RAS], 2014. DOI: 10.15514/ISPRAS-2014-26(1)-5. (in Russian).
38. S. V. Zelenov, A. K. Petrenko, N. V. Pakulin, A. A. Ugnenko, A. V. Khoroshilov. Instrumental'nye sredstva proektirovaniya sistem integrirovannoj modul'noj avioniki [Integrated modular avionics system design tools]. Trudy ISP RАN [Proceedings of ISP RAS], 2014. DOI: 10.15514/ISPRAS-2014-26(1)-6. (in Russian).
39. N. V. Pakulin, A. V. Khoroshilov. Development of Formal Models and Conformance Testing for Systems with Asynchronous Interfaces and Telecommunications Protocols. Programming and Computer Software, 33(6):316-335, 2007.
40. The Linux Foundation consortium. LSB certification test suite, http://ispras.linuxbase.org/index.php/LSB_Certification_System
41. M. M. Chupilko, Developing Test Systems of Multi-Modules Hardware Designs. Programming and Computer Software, 2012, 38(1):34-42, 2012.
42. S. V. Zelenov, S. A. Zelenova. Model-Based Testing of Optimizing Compilers. In: Proc. of the 19th IFIP TC6/WG6.1 International Conference on Testing of Software and Communicating Systems – 7th International Workshop on Formal Approaches to Testing of Software (TestCom/FATES 2007). LNCS, vol. 4581, pp. 365-377. Springer-Verlag, Berlin, 2007.
43. S. V. Zelenov, D. V. Silakov, A. K. Petrenko, M. Conrad, I. Fey. Automatic Test Generation for Model-Based Code Generators. In: IEEE ISoLA 2006 Second Intern. Symposium on Leveraging Applications of Formal Methods, Verification and Validation. Paphos, Cyprus, pp. 68-75, 2006.
44. A. S. Kamkin Metod avtomatizatsii imitatsionnogo testirovaniya mikroprotsessorov konvejernoj arkhitektury na osnove formal'nykh spetsifikatsij [Pipeline microprocessor simulation testing automation based on formal specifications]. PhD Thesis, ISP RAS, Moscow, 2009 (in Russian).
45. E. V. Kornykhin. Metod avtomatizatsii generatsii testovykh programm dlya verifikatsii MMU [Automated generation of etst programs for MMU verification]. PhD Thesis, ISP RAS, Moscow, 2010 (in Russian).
46. A. S. Kamkin, A. D. Tatarnikov. MicroTESK: An ADL-Based Reconfigurable Test Program Generator for Microprocessors. In: Proceedings of the 6th Spring/Summer Young Researchers’ Colloquium on Software Engineering (SYRCoSE 2012), May 30-31, 2012, Perm, Russia, 2012.
Review
For citations:
Kuliamin V., Petrenko A. Evolution of UniTESK Test Development Technology. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2014;26(1):9-26. (In Russ.) https://doi.org/10.15514/ISPRAS-2014-26(1)-1
Email this article 