The Theoretical Approach to the Search for a Global Extremum in the Training of Neural Networks
https://doi.org/10.15514/ISPRAS-2019-31(2)-4
Abstract
The article deals with the search for the global extremum in the training of artificial neural networks using the correlation index. The proposed method is based on a mathematical model of an artificial neural network, represented as an information transmission system. The efficiency of the proposed model is confirmed by its broad application in information transmission systems for analyzing and recovering the useful signal against the background of various interferences: Gaussian, concentrated, pulsed, etc. The analysis of the convergence of training and experimentally obtained sequences based on the correlation index is carried out. The possibility of estimating the convergence of the training and experimentally obtained sequences by the cross-correlation function as a measure of their energy similarity (difference) is confirmed. To evaluate the proposed method, a comparative analysis is carried out with the currently used target indicators. Possible sources of errors of the least squares method and the possibility of the proposed index to overcome them are investigated.
Supporting agencies: Scholarship of the President of the Russian Federation to young scientists and graduate students SP-2236.2018
About the Authors
Nikolay Anatolievitch Vershkov
Stavropol Regional Institute for the Development of Education, Advanced Training and Retraining of Educators
Russian Federation
Russian Federation
Viktor Andreevich Kuchukov
North Caucasus Federal University
Russian Federation
Russian Federation
Natalya Nikolaevna Kuchukova
North Caucasus Federal University
Russian Federation
Russian Federation
References
1. . Kolmogorov A.N. On the representation of continuous functions of several variables in the form of superpositions of continuous functions of one variable and addition. Doklady Akademii nauk, Rossijskaya akademiya nauk [Reports of the Academy of Sciences, Russian Academy of Sciences], vol. 114, № 5, 1957., pp. 953-956. (in Russian)
2. . Arnol'd V.I. On the representation of functions of several variables as a superposition of functions of a smaller number of variables. Mat. Prosveshchenie [Math. education], vol. 3, 1958, pp. 41—61. (in Russian)
3. . Hecht-Nielsen R. Neurocomputing: picking the human brain. IEEE spectrum, vol. 25, no. 3, 1988, pp. 36-41
4. . Dzyadyk V.K. Introduction to the theory of uniform approximation of functions by polynomials. Nauka [Science], 1977. (in Russian)
5. . Hebb D. O. The organization of behavior. New York: Wiley, 1949.
6. . Hinton G.E. Training products of experts by minimizing contrastive divergence. Neural computation, vol. 14, no. 8, 2002, pp. 1771-1800.
7. . Sreenivasulu D., Krishna P.V. Deep Learning Based Efficient Channel Allocation Algorithm for Next Generation Cellular Networks. Programming and Computer Software, vol. 44, no. 6, 2018, 428-434.
8. . Hinton,G.E. Learning multiple layers of representation. Trends in cognitive sciences, vol. 11, no. 10) 2007, pp. 428-434.
9. . Nikolenko S.I., Kadurin A.A., Arhangel'skaya E.O. Deep learning. Piter [Piter], 2018, 480 p. (in Russian)
10. . Shennon K. Works on information theory and cybernetics. Izdatel'stvo inostrannoj literatury [Foreign Literature Publishing House], 1963. (in Russian)
11. . Sikarev A.A., Lebedev O.N. Microelectronic devices for the formation and processing of complex signals. Radio i svyaz' [Radio and communication], 1983. (in Russian)
12. . Widrow B. Adaptive sampled-data systems—a statistical theory of adaptation. IRE Wescon Convention Record, vol. 4, 1959, pp. 74-85.
13. . Ajficher EH.S., Dzhervis B.U. Digital Signal Processing: A Practical Approach, 2nd Edition. Izdatel'skij dom «Vil'yams» [Publishing House "Williams"], 2008, 992 p. (in Russian)
14. . Dorogov A.Y. Implementation of spectral transformations in the class of fast neural networks. Programming and Computer Software, vol. 29, no. 4, 2003, pp.187-198.
15. . Hajkin S. Neural Networks: Full Course, 2nd Edition. Izdatel'skij dom «Vil'yams» [Publishing House "Williams"], 2006, 1104 p. (in Russian)
16. . Adjemov S.S., Klenov N.V., Tereshonok M.V., Chirov D.S. The use of artificial neural networks for classification of signal sources in cognitive radio systems. Programming and Computer Software, vol. 42, no. 3, 2016, pp 121–128.
17. . Solodov A.V. Information theory and its application to the tasks of automatic control and monitoring. Izd-vo "Nauka", glav. red. fiziko-matematicheskoj lit-ry [Publishing house "Science"], 1967.
18. . Linnik YU. V. The method of least squares and the basics of mathematical and statistical theory of processing observations. Gos. izd-vo fiziko-matematicheskoj lit-ry [State publishing house of physical and mathematical literature], 1958 (in Russian)
Review
For citations:
Vershkov N.A., Kuchukov V.A., Kuchukova N.N. The Theoretical Approach to the Search for a Global Extremum in the Training of Neural Networks. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2019;31(2):41-52. (In Russ.) https://doi.org/10.15514/ISPRAS-2019-31(2)-4
Email this article 