Vehicle Image Classifier for Bridge Displacement Correlation
https://doi.org/10.15514/ISPRAS-2021-33(2)-8
Abstract
Advanced computing brings opportunities for innovation in a broad gamma of applications. Traditional practices based on visual and manual methods tend to be replaced by cyber-physical systems to automate processes. The present work introduces an example of this, a machine vision system research based on deep learning to classify bridge load, to give support to an optical scanning system for structural health monitoring tasks. The optical scanning system monitors the health of structures, such as buildings, warehouses, water dams, etc. by the measurement of their coordinates to identify if a coordinate displacement befalls that could indicate an anomaly in the structure that can be related to structural damage. The use of this optical scanning system to monitor the structural health of bridges is a little more complicated due to the vehicle's transit over the bridge that causes a vehicle-bridge interaction which manifests as a bridge oscillation. Under this scheme, the bridge oscillation corresponds to their coordinate’s displacement due to the vehicle-bridge interaction, but not necessarily due to bridge damage. So, a bridge load classifier is required to correlate the bridge coordinates measurements behavior with the bridge oscillation due to vehicle-bridge interaction to discriminate the normal behavior of the structure to abnormal behavior or identify tendencies that could indicate bridge deformation or discover if the bridge behavior due to loads is changing through the time.
About the Author
Wendy FLORES-FUENTES
Autonomous University of Baja California
Mexico
Mexico
Doctor in Sciences, Research-Professor at Engineering Faculty
References
1. R. Massobrio, S. Nesmachnow, A. Tchernykh et al. Towards a cloud computing paradigm for big data analysis in smart cities. Programming and Computer Software, vol. 44, no. 3, 2018, pp. 181-189.
2. Р. Массобрио, С. Несмачнов, А. Черных и др. Применение облачных вычислений для анализа данных большого объема в умных городах. Труды ИСП РАН, том 28, вып. 6, 2016 г., стр. 121-140 / R. Massobrio, S. Nesmachnow, A. Tchernykh et al. Towards a Cloud Computing Paradigm for Big Data Analysis in Smart Cities. Trudy ISP RAN/Proc. ISP RAS, vol. 28, issue 6, 2016. pp. 121-140 (in Russian). DOI: 10.15514/ISPRAS-2016-28(6)-9.
3. A. Mufti, K. Helmi. A case for structural health monitoring (SHM) and civionics enhances the evaluation of the load carrying capacity of aging bridges. Innovative Infrastructure Solutions, vol. 4, no. 1, pp. 3, 2019.
4. M. Barousse Moreno, A. Galindo Solozano. Sistema de Administracion de Puentes (SIAP). Instituto Mexicano del Transporte, Publicación Técnica, vol. 49, 1994, 88 p. (in Spanish).
5. W. Flores-Fuentes, M. Rivas-Lopez, O. Sergiyenko et al. Energy center detection in light scanning sensors for structural health monitoring accuracy enhancement. IEEE Sensors Journal, vol. 14, no. 7, 2014, pp. 2355-2361.
6. J. Rivera-Castillo, W. Flores-Fuentes, M. Rivas‐López et al. Experimental image and range scanner datasets fusion in SHM for displacement detection. Structural Control and Health Monitoring, vol. 24, no. 10, 2017, article e1967.
7. L. Ma, W. Zhang, W. S. Han, & J. X. Liu. Determining the dynamic amplification factor of multi-span continuous box girder bridges in highways using vehicle-bridge interaction analyses. Engineering Structures, vol. 181, 2019, pp. 47-59.
8. C. Emmanouilidis, P. Pistofidis, L. Bertoncelj et al. Enabling the human in the loop: Linked data and knowledge in industrial cyber-physical systems. Annual Reviews in Control, vol. 47, 2019, pp. 249-265.
9. P. Ramachandran, B. Zoph, & Q.V. Le. Swish: a self-gated activation function. arXiv preprint arXiv:1710.05941, 2017.
10. Y. Ren, J. Huang, Z. Hong et al. Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction and Building Materials, vol. 234, 2020, article no. 117367.
11. N. S. Gulgec, M. Takáč, & S. N. Pakzad. Experimental Study on Digital Image Correlation for Deep Learning-Based Damage Diagnostic. In Dynamics of Civil Structures, vol. 2, Springer, 2020, pp. 205-210.
12. W. Deng, Y. Mou, T. Kashiwa et al. Vision based pixel-level bridge structural damage detection using a link ASPP network. Automation in Construction, vol. 110, 2020, article no. 102973.
13. J. J. Rubio, T. Kashiwa, T. Laiteerapong et al. Multi-class structural damage segmentation using fully convolutional networks. Computers in Industry, vol. 112, 2019, article no. 103121.
14. W. Deng, Y. Mou, T. Kashiwa et al. Vision based pixel-level bridge structural damage detection using a link ASPP network. Automation in Construction, vol. 110, 2020, article no. 102973.
15. Y. Bao, Z. Tang, H. Li, & Y. Zhang. Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Structural Health Monitoring, vol. 18, vol. 2, 2019, pp. 401-421.
16. Y. Xia, X. Jian, B. Yan, & D. Su. Infrastructure Safety Oriented Traffic Load Monitoring Using Multi-Sensor and Single Camera for Short and Medium Span Bridges. Remote Sensing, vol. 11, no. 22, 2019, article no. 2651.
17. X. Jian, Y. Xia, J. A. Lozano-Galant, & L. Sun. Traffic Sensing Methodology Combining Influence Line Theory and Computer Vision Techniques for Girder Bridges. Journal of Sensors, Special Issue, 2019, Article ID 3409525.
18. D.K. Amara, R. Karthika, & K.P. Soman. DeepTrackNet: Camera Based End to End Deep Learning Framework for Real Time Detection, Localization and Tracking for Autonomous Vehicles. Advances in Intelligent Systems and Computing book series, vol. 1039, 2019, pp. 299-307.
19. C. Cardenas, & M. Gonzalez-Mendoza. Distributed System Based on Deep Learning for Vehicular Re-routing and Congestion Avoidance. Advances in Intelligent Systems and Computing book series, vol. 1071, 2019, pp. 159-172.
20. S. Zhang, C. Wang, Z. He et al. Vehicle global 6-DoF pose estimation under traffic surveillance camera. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 159, 2020, pp. 114-128.
21. C.K. Ng, S.N. Cheong, & Y.L. Foo. Low Latency Deep Learning Based Parking Occupancy Detection By Exploiting Structural Similarity. Lecture Notes in Electrical Engineering, vol. 603, 2020, pp. 247-256.
22. J. E. Miranda-Vega, W. Flores-Fuentes, O. Sergiyenko et al. Optical cyber-physical system embedded on an FPGA for 3D measurement in structural health monitoring tasks. Microprocessors and Microsystems, vol. 56, 2018, pp. 121-133.
23. V. Tyrsa, O. Sergiyenko, L. Burtseva et al. Mobile transport object control by technical vision means. In Proc. of the Electronics, Robotics and Automotive Mechanics Conference (CERMA'06), vol. 2, 2005, pp. 74-82.
24. М.В. Иванов, О.Ю. Сергиенко, В.В. Тырса и др. Интеграция беспроводной связи для оптимизации распознавания окружения и расчёта траектории движения группы роботов. Труды ИСПРАН, том 31, вып. 2, 2019 г., стр. 67-82. DOI: 10.15514/ISPRAS-2019-31(2)-6 / M. Ivanov, O. Sergiyenko, V. Tyrsa et al. Software Advances using n-agents Wireless Communication Integration for Optimization of Surrounding Recognition and Robotic Group Dead Reckoning. Programming and Computer Software, vol. 45, no. 8, 2019, pp. 557-569.
25. M. Rivas, O. Sergiyenko, M. Aguirre et al. Spatial data acquisition by laser scanning for robot or SHM task. In Proc. of the 2008 IEEE International Symposium on Industrial Electronics, 2008, pp. 1458-1462.
26. M. R. López, O. Sergiyenko, & V. Tyrsa. Machine vision: approaches and limitations. In Computer vision. IntechOpen, 2008, pp. 395-428.
27. O. Sergiyenko, V. Tyrsa, D. Hernandez-Balbuena et al. Precise optical scanning for practical multi-applications. In Proc. of the 2008 34th Annual Conference of IEEE Industrial Electronics (pp. 1656-1661). IEEE. 2008, November.
28. W. Flores-Fuentes, M. Rivas-Lopez, O. Sergiyenko et al. Energy center detection in light scanning sensors for structural health monitoring accuracy enhancement. IEEE Sensors Journal, vol. 14, no. 7, 2014, pp. 2355-2361.
29. W. Flores-Fuentes, M. Rivas-Lopez, O. Sergiyenko et al. Combined application of power spectrum centroid and support vector machines for measurement improvement in optical scanning systems. Signal Processing, vol. 98, 2014, pp. 37-51.
30. M. Reyes-Garcia, C. Sepulveda-Valdez, O. Sergiyenko et al. Digital Control Theory Application and Signal Processing in a Laser Scanning System Applied for Mobile Robotics. In Control and Signal Processing Applications for Mobile and Aerial Robotic Systems, IGI Global, 2020, pp. 215-265.
31. L. Lindner, O. Sergiyenko, M. Rivas-López et al. Machine vision system errors for unmanned aerial vehicle navigation. In Proc. of the 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE), 2017, pp. 1615-1620.
32. W. Flores-Fuentes, J. Miranda-Vega, M. Rivas-López et al. Comparison between Different Types of Sensors Used in the Real Operational Environment Based on Optical Scanning System. Sensors, vol. 18, no. 6, 2018, article no. 1684.
33. J. Krause, M. Stark, J. Deng, & L. Fei-Fei.. 3d object representations for fine-grained categorization. In Proc. of the IEEE International Conference on Computer Vision Workshops, 2013, pp. 554-561.
34. Н.П. Варновский, С А. Мартишин, М В. Храпченко, А.В. Шокуров. Методы пороговой криптографии для защиты облачных вычислений. Труды ИСП РАН, том 26, вып. 2, 2015 г., стр. 269-274 / N.P. Varnovskiy, S.A. Martishin, M.V. Khrapchenko, & A.V. Shokurov (2015). Secure cloud computing based on threshold homomorphic encryption. Programming and Computer Software, vol. 41, no. 4, 2015, pp. 215-218.
35. Miranda-López, V., Tchernykh, A., Cortés-Mendoza et al. Experimental analysis of secret sharing schemes for cloud storage based on RNS. Communications in Computer and Information Science (CCIS), vol. 79, 2017, pp. 370-383.
36. A. Tchernykh, M. Babenko, N. Chervyakov et al. Towards mitigating uncertainty of data security breaches and collusion in cloud computing. In Proc. of the 28th International Workshop on Database and Expert Systems Applications (DEXA), 2017, pp. 137-141.
37. A. Tchernykh, M. Babenko, N Chervyakov et al. AC-RRNS: Anti-collusion secured data sharing scheme for cloud storage. International Journal of Approximate Reasoning, vol. 102, 2018, pp. 60-73.
Review
For citations:
FLORES-FUENTES W. Vehicle Image Classifier for Bridge Displacement Correlation. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2021;33(2):137-148. https://doi.org/10.15514/ISPRAS-2021-33(2)-8
Email this article 