Method for Convolutional Neural Network Hardware Implementation Based on a Residue Number System
Maria Vasilyevna VALUEVA,
Georgii Vyacheslavovich VALUEV,
Mikhail BABENKO,
Andrei TCHERNYKH,
Jorge Mario CORTES-MENDOZA https://doi.org/10.15514/ISPRAS-2022-34(3)-5
Abstract
Convolutional Neural Networks (CNN) show high accuracy in pattern recognition solving problem but have high computational complexity, which leads to slow data processing. To increase the speed of CNN, we propose a hardware implementation method with calculations in the residue number system with moduli of a special type and . A hardware simulation of the proposed method on Field-Programmable Gate Array for LeNet-5 CNN is trained with the MNIST, FMNIST, and CIFAR-10 image databases. It has shown that the proposed approach can increase the clock frequency and performance of the device by 11%-12%, compared with the traditional approach based on the positional number system.
Supporting agencies: This work was supported in part by the Russian Science Foundation, project number 19-71-10033
About the Authors
Maria Vasilyevna VALUEVA
North-Caucasus Center for Mathematical Research NCFU
Russian Federation
Russian Federation
Junior Researcher
Georgii Vyacheslavovich VALUEV
North-Caucasus Center for Mathematical Research NCFU
Russian Federation
Russian Federation
Junior Researcher
Mikhail BABENKO
North-Caucasus Federal University,
Ivannikov Institute for System Programming of the Russian Academy of Sciences
Russian Federation
Russian Federation
Ph.D. in Physics and Mathematics
Andrei TCHERNYKH
Ivannikov Institute for System Programming of the Russian Academy of Sciences,
CICESE Research Center, Mexico,
South Ural State University
Mexico
Mexico
Received doctor of science degree at Ivannikov Institute for System Programming of the Russian Academy of Sciences, is holding a full professor position in computer science at CICESE Research Center, Ensenada, Baja California, Mexico
Jorge Mario CORTES-MENDOZA
South Ural State University
Russian Federation
Russian Federation
Received Ph.D. degree in 2018 from CICESE Research Center in Computer Scienc
References
1. Ashiq F., Asif M. et al. CNN-Based Object Recognition and Tracking System to Assist Visually Impaired People. IEEE Access, vol. 10, 2022, pp. 14819-14834.
2. Moon C.-I., Lee O. Skin Microstructure Segmentation and Aging Classification Using CNN-Based Models. IEEE Access, vol. 10, 2022, pp. 4948-4956.
3. Mondal A.K., Bhattacharjee A. et al. xViTCOS: Explainable Vision Transformer Based COVID-19 Screening Using Radiography. IEEE Journal of Translational Engineering in Health and Medicine, vol. 10, 2022, article no. 1100110, pp. 1-10.
4. Elharrouss O., Almaadeed N. et al. FSC-Set: Counting, Localization of Football Supporters Crowd in the Stadiums. IEEE Access, vol. 10, 2022, pp. 10445-10459.
5. Vieira J.C., Sartori A. et al. Low-cost CNN for Automatic Violence Recognition on Embedded System. IEEE Access, vol. 10, 2022, pp. 25190-25202.
6. Wong C.-C., Chien M.-Y. et al. Moving Object Prediction and Grasping System of Robot Manipulator. IEEE Access, vol. 10, 2022, pp. 20159-20172.
7. Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. In Proc. of the 25th International Conference on Neural Information Processing Systems, vol. 1, 2012, pp, 1097–1105.
8. Nakahara H., Sasao T. A deep convolutional neural network based on nested residue number system. In Proc. of the 25th International Conference on Field Programmable Logic and Applications (FPL), 2015, pp. 1-6.
9. Nakahara H., Sasao T. A High-speed Low-power Deep Neural Network on an FPGA based on the Nested RNS: Applied to an Object Detector. In Proc. of IEEE International Symposium on Circuits and Systems (ISCAS), 2018, pp. 1-5.
10. Salamat S., Imani M. et al. RNSnet: In-Memory Neural Network Acceleration Using Residue Number System. In Proc. of IEEE International Conference on Rebooting Computing (ICRC), 2018, pp. 1-12.
11. Omondi A., Premkumar B. Residue Number Systems: Theory and Im-plementation. London, Imperial College Press, 2007. 296 p.
12. Chervyakov N.I., Lyakhov P.A. et al. Residue Number System-Based Solution for Reducing the Hardware Cost of a Convolutional Neural Network. Neurocomputing, vol. 407, 2020, pp. 439-453.
13. Parhami B. Computer arithmetic: algorithms and hard-ware designs. Oxford University Press, 2010. 492 p.
14. Vergos H.T., Dimitrakopoulos G. On Modulo 2^n+1 Adder Design. IEEE Transactions on Computers, vol. 61, issue 2, 2012, pp. 173-186.
15. Kogge P.M., Stone H.S. A Parallel Algorithm for the Efficient Solution of a General Class of Recurrence Equations. IEEE Transactions on Computers, vol. C-22, issue 8, 1973, pp. 786–793.
16. Chervyakov N.I., Lyakhov P.A., Valueva M.V. Increasing of Convolutional Neural Network Performance Using Residue Number System. In Proc. of the International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), 2017, pp. 135–140.
17. Tung C., Huang S.A High-Performance Multiply-Accumulate Unit by Integrating Additions and Accumulations into Partial Product Reduction Process. IEEE Access, vol. 8, 2020, pp. 87367-87377.
18. Habibi Aghdam H., Jahani Heravi E. Guide to Convolutional Neural Networks. Springer, 2017, 305 p.
19. Valueva M., Valuev G. et al. Construction of Residue Number System Using Hardware Efficient Diagonal Function. Electronics, vol. 8, issue 6, 2019, article no 694, pp. 1-16.
20. Chervyakov N.I., Molahosseini A.S. et al. Residue-to-binary conversion for general moduli sets based on approximate Chinese remainder theorem. International Journal of Computer Mathematics, vol. 94, issue 9, 2017, pp. 1833–1849.
21. Haykin S.S. Neural networks: a comprehensive foundation. Prentice Hall, 1999. 842 p.
22. LeCun Y., Bottou L. et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, vol. 86, issue 11, 1998, pp.2278-2324.
23. Abadi M., Agarwal A. et al. TensorFlow: Large-scale machine learning on heterogeneous systems. arXiv preprint arXiv:1603.04467, 1916, 19 p.
24. Xiao H., Kashif R., Vollgraf R. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017, 6 p.
25. Krizhevsky A. Learning Multiple Layers of Features from Tiny Images. Technical Report TR-2009, University of Toronto, 2009, 60 p.
Review
For citations:
VALUEVA M.V., VALUEV G.V., BABENKO M., TCHERNYKH A., CORTES-MENDOZA J. Method for Convolutional Neural Network Hardware Implementation Based on a Residue Number System. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2022;34(3):61-74. (In Russ.) https://doi.org/10.15514/ISPRAS-2022-34(3)-5
Email this article 