Реализация искусственных нейронных сетей на ПЛИС с помощью открытых инструментов

Искусственные нейронные сети широко распространены в современном мире. Для их исполнения используются различные устройства: от микропроцессоров до ПЛИС и заказных СБИС. Важной проблемой при этом является ускорение исполнения нейронных сетей. В этой области на данный момент существует множество открытых инструментов. В данной статье содержится обзор нескольких открытых инструментов для исполнения, ускорения нейронных сетей и синтеза аппаратуры по ним. Некоторые из рассмотренных инструментов были выбраны для апробации на ПЛИС. Для этого было разработано пять тестовых моделей нейронных сетей. Процессор Intel, графический процессор NVIDIA и ПЛИС Cyclone V использовались для проведения экспериментов. Результаты показали, что инструменты TVM/VTA и LeFlow оказались способны довести тестовые модели до исполнения на ПЛИС. Однако результаты исполнения показали, что в большинстве случаев ПЛИС проигрывает в быстродействии другим платформам.

1. TensorFlow framework. Available at: https://www.tensorflow.org, accessed 21.10.2021.

2. PyTorch framework. Available at: https://pytorch.org, accessed 21.10.2021.

3. Caffe framework. Available at: https://caffe.berkeleyvision.org, accessed 21.10.2021.

4. ONNX format. Available at: https://onnx.ai, accessed 21.10.2021.

5. Lu Y., Gong L. et al. A high-performance FPGA accelerator for sparse neural networks: work-in-progress. In Proc. of the 2017 International Conference on Compilers, Architectures and Synthesis for Embedded Systems Companion, 2017, pp. 1-2.

6. Sigurbergsson B., Hogervorst T. et al. Sparstition: A Partitioning Scheme for Large-Scale Sparse Matrix Vector Multiplication on FPGA. In Proc. of the IEEE 30th International Conference on Application-specific Systems, Architectures and Processors, 2019, pp. 51-58.

7. Yang L., He Z., Fan D. A Fully Onchip Binarized Convolutional Neural Network FPGA Implementation with Accurate Inference. In Proc. of the International Symposium on Low Power Electronics and Design, 2018, pp. 1-6.

8. Chi C., Jiang J. R. Logic Synthesis of Binarized Neural Networks for Efficient Circuit Implementation. In Proc. of the IEEE/ACM International Conference on Computer-Aided Design, 2018, pp. 1-7.

9. Zhuge C., Liu X. et al. Face Recognition with Hybrid Efficient Convolution Algorithms on FPGAs. In Proc. of the Great Lakes Symposium on VLSI, 2018, pp. 123-128.

10. Xiao Q., Liang Y. et al. Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs. In Proc. of the 54th Annual Design Automation Conference, 2017, pp. 1-6.

11. Lu L., Liang Y. SpWA: an efficient sparse Winograd convolutional neural networks accelerator on FPGAs. In Proceedings of the 55th Annual Design Automation Conference, 2018, pp. 1-6.

12. Alwani M., Chen H. et al. Fused-layer CNN accelerators. In Proc. of the 49th Annual International Symposium on Microarchitecture, 2016, pp. 1-12.

13. Samragh M., Javaheripi M., Koushanfar F.F. EncoDeep: Realizing Bit-flexible Encoding for Deep Neural Networks. ACM Transactions on Embedded Computing Systems, vol. 19, issue 6, 2020, Article 43, 29 p.

14. Ding C., Wang S. et al. REQ-YOLO: A Resource-Aware, Efficient Quantization Framework for Object Detection on FPGAs. In Proc. of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2019, pp. 33-42.

15. Karki A., Keshava C. P. et al. Detailed Characterization of Deep Neural Networks on GPUs and FPGAs. In Proc.of the 12th Workshop on General Purpose Processing Using GPUs, 2019, pp. 12-21.

16. Mittal S., Umesh S. A survey on hardware accelerators and optimization techniques for RNNs. Journal of Systems Architecture, vol. 112, 2020, article no. 101839.

17. Kouris A., Venieris S. I., Bouganis C.-S. A throughput-latency co-optimised cascade of convolutional neural network classifiers. In Proc. of the 23rd Conference on Design, Automation and Test in Europe, 2020, pp. 1656-1661.

18. Lin X., Yin S. et al. LCP: a layer clusters paralleling mapping method for accelerating inception and residual networks on FPGA. In Proc. of the 55th Annual Design Automation Conference, 2018, pp. 1-6.

19. Keras framework. Available at: https://keras.io, accessed 21.10.2021.

20. Bücker H. M., Corliss G. et al., editors. Automatic Differentiation: Applications, Theory, and Implementations. Lecture Notes in Computational Science and Engineering, vol. 50, Springer, 2006, 361 p.

21. Caffe2 framework. Available at: https://caffe2.ai, accessed 21.10.2021.

22. CNTK framework. Available at: https://github.com/Microsoft/CNTK, accessed 21.10.2021.

23. CoreML framework. Available at: https://coremltools.readme.io, accessed 21.10.2021.

24. MXNet framework. Available at: https://mxnet.apache.org, accessed 21.10.2021.

25. Theano framework. Available at: https://github.com/Theano/Theano, accessed 21.10.2021

26. Aesara library. Available at: https://github.com/aesara-devs/aesara, accessed 21.10.2021

27. LegUp 4.0 high-level synthesis tool. Available at: http://legup.eecg.utoronto.ca, accessed 30.12.2020.

28. Vivado Design Suite. Available at: https://www.xilinx.com/products/design-tools/vivado.html, accessed 21.10.2021.

29. OpenVINO toolkit. Available at: https://github.com/openvinotoolkit/openvino, accessed 21.10.2021.

30. Kaldi Speech Recognition Toolkit. Available at: https://kaldi-asr.org, accessed 21.10.2021.

31. PlaidML framework. Available at: https://github.com/plaidml/plaidml, accessed 21.10.2021.

32. nGraph compiler stack. Available at: https://github.com/NervanaSystems/ngraph, accessed 21.10.2021.

33. MACE framework. Available at: https://github.com/XiaoMi/mace, accessed 21.10.2021.

34. Apache TVM ML compiler framework. Available at: https://tvm.apache.org, accessed 21.10.2021.

35. VTA: Deep Learning Accelerator Stack. Available at: https://tvm.apache.org/docs/vta/index.html

36. XLA neural network compiler. Available at: https://www.tensorflow.org/xla, accessed 21.10.2021.

37. The Julia programming language. Available at: https://julialang.org, accessed 21.10.2021.

38. JAX differentiation library. Available at: https://github.com/google/jax, accessed 21.10.2021.

39. Nx multidimensional array representation library. Available at: https://github.com/elixir-nx/nx, accessed 21.10.2021.

40. LeFlow tool-flow. Available at: https://github.com/danielholanda/LeFlow, accessed 21.10.2021.

41. Glow ML compiler. Available at: https://ai.facebook.com/tools/glow, accessed 21.10.2021.

42. hls4ml tool. Available at: https://github.com/fastmachinelearning/hls4ml, accessed 21.10.2021.

43. Kreinar E. Rfnoc Neural Network Library Using Vivado HLS. In Proc. of the GNU Radio Conference, 2017, 7 p.

44. NNgen hardware synthesis compiler. Available at: https://github.com/NNgen/nngen, accessed 21.10.2021.

45. ONNC compilation framework. Available at: https://onnc.ai, accessed 21.10.2021.

46. Xilinx Vitis AI development stack for AI inference. Available at: https://github.com/Xilinx/Vitis-AI, accessed 21.10.2021.

47. Xilinx runtime for FPGA. Available at: https://github.com/Xilinx/XRT, accessed 21.10.2021.

48. FINN framework. Available at: https://xilinx.github.io/finn, accessed 21.10.2021.

49. Vivado HLS library for FINN. Available at: https://github.com/Xilinx/finn-hlslib, accessed 21.10.2021.

50. Terasic DE10-Standard board. Available at: https://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=165&No=1081. accessed 21.10.2021.

51. Terasic DE1-SoC board. Available at: https://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=165&No=836, accessed 21.10.2021.

52. Zybo Z7-20 board. Available: https://www.xilinx.com/products/boards-and-kits/1-pukio3.html, accessed 21.10.2021.

53. Quartus Prime Software Suite. Available at: https://www.intel.com/content/www/us/en/software/programmable/quartus-prime/overview.html, accessed 21.10.2021.

54. Linux development repository for socfpga. Available at: https://github.com/altera-opensource/linux-socfpga/tree/socfpga-4.14.130-ltsi, accessed 21.10.2021.

55. Ubuntu Base rootfs. Available at: https://wiki.ubuntu.com/Base, accessed 21.10.2021.

56. Linux drivers (modules) for Field Programmable Systems on Chip. Available at: http://git.edi.lv/rihards.novickis/FPSoC_Linux_drivers, accessed 21.10.2021.