An Automated Framework for Testing Source Code Static Analysis Tools
Damir Maratovich GIMATDINOV,
Alexander Yurievich GERASIMOV,
Petr Alekseevich PRIVALOV,
Veronika Nikolaevna BUTKEVICH,
Natalya Andreevna CHERNOVA,
Anna Antonovna GORELOVA https://doi.org/10.15514/ISPRAS-2021-33(3)-3
Abstract
Automated testing frameworks are widely used for assuring quality of modern software in secure software development lifecycle. Sometimes it is needed to assure quality of specific software and, hence specific approach should be applied. In this paper, we present an approach and implementation details of automated testing framework suitable for acceptance testing of static source code analysis tools. The presented framework is used for continuous testing of static source code analyzers for C, C++ and Python programs.
About the Authors
Damir Maratovich GIMATDINOV
HSE University, Huawei Technologies
Russian Federation
Russian Federation
HSE graduate, master, Junior engineer in Huawei Technologies
Alexander Yurievich GERASIMOV
Huawei Technologies
Russian Federation
Russian Federation
Doctor of Philosophy in Computer Sciences, Senior Expert in the field of automatic and automated analysis of electronic computer programs in Huawei Technologies
Petr Alekseevich PRIVALOV
Huawei Technologies
Russian Federation
Russian Federation
Master, Senior software engineer
Veronika Nikolaevna BUTKEVICH
Huawei Technologies
Russian Federation
Russian Federation
Master, developer
Natalya Andreevna CHERNOVA
Huawei Technologies
Russian Federation
Russian Federation
Master, junior developer
Anna Antonovna GORELOVA
Huawei Technologies
Russian Federation
Russian Federation
Junior Developer
References
1. M. Cooper. Advanced Bash Scripting Guide – Volume 1: An in-depth exploration of the art of shell scripting. (Revision 10). Independently published, 2019, 589 p.
2. M.-A. Lemburg, M. von Löwis. PEP-263 – Defining Python Source Code Encodings. 2001. URL: https://www.python.org/dev/peps/pep-0263/.
3. NIST SAMATE Juliet Test Suite. URL: https://samate.nist.gov/SRD/testsuite.php.
4. RFC-8259. The JavaScript Object Notation (JSON) Data Interchange Format, 2017. URL: https://datatracker.ietf.org/doc/html/rfc8259.
5. H.H. AlBreiki, Q.H. Mahmoud. Evaluation of static analysis tools for software security. In Proc. of the IEEE 2014 10th International Conference on Innovations in Information Technology, 2014, pp. 93-98,
6. R. Mamood, Q.H. Mahmoud. Evaluation of static analysis tools for finding vulnerabilitites in Java and C/C++ source code. arXiv:1805.09040, 2018, 7 p.
7. T. Hofer. Evaluating static source code analysis tools. Master’s thesis. École Polytechnique Fédérale de Lausanne, 2010, pp. 1-74.
8. OWASP – Open web application security project. URL: https://owasp.org
9. M. Johns, M. Jodeit. Scanstud: a methodology for systematic, fine-grained, evaluation of static analysis tools. 4th International conference on software testing, verification and validation workshops. In Proc. of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, 2011, pp. 523-530.
10. G. Hao, F. Li et al. Constructing benchmarks for supporting explainable evaluations of static application security testing tools. In Proc. of the 2019 International symposium on Theoretical Aspects of Software Engineering, 2019, pp. 66-72.
11. H.G. Rice. Classes of Recursively Enumerable Sets and Their Decision Problems. Transactions of the American Mathematical Society, vol. 74, no. 2, 1953, pp. 358-366.
12. Pylint. URL: https://pypi.org/project/pylint/.
13. JetBrains PyCharm. URL: https://www.jetbrains.com/pycharm/.
14. Flake8. URL: https://pypi.org/project/flake8/.
15. Pep8 – Python style guide checker. URL: https://pypi.org/project/pep8/.
16. Pyflakes. URL: https://github.com/PyCQA/pyflakes.
17. McCabe complexity checker. URL: https://github.com/PyCQA/mccabe.
Review
For citations:
GIMATDINOV D.M., GERASIMOV A.Yu., PRIVALOV P.A., BUTKEVICH V.N., CHERNOVA N.A., GORELOVA A.A. An Automated Framework for Testing Source Code Static Analysis Tools. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2021;33(3):41-50. https://doi.org/10.15514/ISPRAS-2021-33(3)-3
Email this article 