Automatic API fuzzing framework

Randomized testing (fuzzing) is a well-known approach for finding bugs in programs. Fuzzing is typically performed during the finishing stage of quality assurance in order to check the stability of the target program in the face of malformed or unexpected input data. Modern software more than often provides an API for extending its functionality by third-party developers; since an API is an entry point to software internals, its functionality and usage scenarios must be tested as well. Thorough API testing must involve checking a large number of possible scenarios and it is fairly obvious that fuzzing can be applied to this task by generating usage scenarios in an automatic randomized way—which brings us to the concept of API fuzzing. In this paper we describe an automatic approach to randomized testing of API libraries for Android/desktop Java. Proposed method is able to change the sequence of called API functions in order to discover new execution paths. It consists of two basic stages. In the first stage the arguments of currently called API functions are mutated. When mutation of called API functions arguments can’t find new execution path the tool switches to the second stage. In the second stage current sequence of API functions calls is mutated. Mutation can add new API functions calls or remove some of them. After API calls sequence mutation, the tool switches back to the first stage. Switches between the first and the second stages are continued during whole process of fuzzing. During the experimental setup developed method of randomized testing were able to find 15 crashes in SmartThings application developed by Samsung.

1. SDL. Available at: https://www.microsoft.com/en-us/securityengineering/sdl, accessed 26.11.2019.

2. Fuzzingю Available at: https://en.wikipedia.org/wiki/Fuzzing, accessed 26.11.2019.

3. Borton P. Miller, Lars Fedriksen. Bryan So. An Empirical Study of the Reliability of UNIX utilities. Communications of the ACM, vol. 33, № 12, 1990, pp. 32-44.

4. Unix. Available at: https://en.wikipedia.org/wiki/Unix, accessed 26.11.2019.

5. Michael Zelewski. American Fuzzy Lop. 2014. Available at: http://lcamtuf.coredump.cx/afl. Accessed 26.11.2019.

6. Sevak Sargsyan, Shamil Kurmangaleev, Matevos Mehrabyan, Maksim Mishechkin, Tsolak Ghukasyan, Sergey Asryan. Grammar-Based Fuzzing. In Proc. of the 2018 Ivannikov Memorial Workshop (IVMEM), Yerevan, Armenia, 2018, pp. 32-35, doi: 10.1109/IVMEM.2018.00013.

7. Sevak Sargsyan, Shamil Kurmangaleev, Jivan Hakobyan, Matevos Mehrabyan, Sergey Asryan, Hovhannes Movsisyan. Directed Fuzzing Based on Program Dynamic Instrumentation. In Proc. of the 2019 International Conference on Engineering Technologies and Computer Science (EnT), Moscow, Russia, 2019, pp. 30-33, doi: 10.1109/EnT.2019.00011.

8. Gerasimov A.Yu., Sargsyan S.S., Kurmangaleev S.F., Hakobyan J.A., Asryan S.A., Ermakov M.K. Combining dynamic symbolic execution and fuzzing. Trudy ISP RAN/Proc. ISP RAS, vol. 30, issue 6, 2018, pp. 25-38. DOI: 10.15514/ISPRAS-2018-30(6)-2.

9. libFuzzer. Available at: https://llvm.org/docs/LibFuzzer.html, accessed 26.11.2019.

10. Trinity: Linux system call fuzzer. Available at: https://github.com/kernelslacker/trinity, accessed 26.11.2019.

11. syzkaller: Linux syscall fuzzer. Available at: https://github.com/google/syzkaller syzkaller, accessed 26.11.2019.

12. Linux. Available at: https://www.linux.org/, accessed 26.11.2019.

13. JAR. Available at: https://en.wikipedia.org/wiki/JAR_(file_format), accessed 26.11.2019.

14. Sevak Sargsyan, Jivan Hakobyan, Matevos Mehrabyan, Maxim Mishechkin, Vitaliy Akozin, Shamil Kurmangaleev. ISP-Fuzzer: Extendable Fuzzing Framework. In Proc. of the 2019 Ivannikov Memorial Workshop (IVMEM), Velikiy Novgorod, Russia, 2019, pp. 68-71, doi: 10.1109/IVMEM.2019.00017.

15. SmartThings. Available at: https://www.samsung.com/global/galaxy/apps/smartthings/, accessed 26.11.2019.

16. Samsung. Available at: https://www.samsung.com, accessed 26.11.2019.

17. Bruneton E., Lenglet R., Coupaye T. ASM: A code manipulation tool to implement adaptable systems. In Proc. of the ASF (ACM SIGOPS France) Journées Composants 2002: Syst`emes `a composants adaptables et extensibles (Adaptable and Extensible Component Systems), Grenoble, France, 2002.

18. Gordon Fraser, Andrea Arcuri. EvoSuite: automatic test suite generation for object-oriented software. In Proc. of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, Szeged, Hungary, 2011, pp. 416-419.

19. JUnit4. Available at: https://junit.org/junit4/, accessed 26.11.2019.

20. Android. Available at: https://www.android.com/, accessed 26.11.2019.

21. Christoph Csallner, Yannis Smaragdakis, Tao Xie. DSD-Crasher: A hybrid analysis tool for bug finding. ACM Transactions on Software Engineering and Methodology, vol.17, issue 2, 2008, Article No. 8.

22. Christoph Csallner, Yannis Smaragdakis. JCrasher: an automatic robustness tester for Java. Software – Practice & Experience, vol. 34, issue 11, 2004, pp. 1025 - 1050.

23. Carlos Pacheco, Michael D. Ernst. Randoop: feedback-directed random testing for Java. In Proc.of the OOPSLA '07 Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion, Montreal, Quebec, Canada, 2007, pp. 815-816.