Methods of gait recognition in video
https://doi.org/10.15514/ISPRAS-2019-31(1)-5
Abstract
Human gait is an important biometric index that allows to identify a person at a great distance without direct contact. Due to these qualities, which other popular identifiers such as fingerprints or iris do not have, the recognition of a person by the manner of walking has become very common in various areas where video surveillance systems can be used. With the development of computer vision techniques, a variety of approaches for human identification by movements in a video appear. These approaches are based both on natural biometric characteristics (human skeleton, silhouette, and their change during walking) and abstract features trained automatically which do not have physical justification. Modern methods combine classical algorithms of video and image analysis and new approaches that show excellent results in related tasks of computer vision, such as human identification by face and appearance or action and gesture recognition. However, due to the large number of conditions that can affect the walking manner of a person itself and its representation in video, the problem of identifying a person by gait still does not have a sufficiently accurate solution. Many methods are overfitted by the conditions presented in the databases on which they are trained, which limits their applicability in real life. In this paper, we provide a survey of state-of-the-art methods of gait recognition, their analysis and comparison on several popular video collections and for different formulations of the problem of recognition. We additionally reveal the problems that prevent the final solution of gait identification challenge.
About the Authors
Anna Ilyinichna Sokolova
National research University Higher School of Economics
Russian Federation
Russian Federation
Anton Sergeevitch Konushin
Lomonosov Moscow State University
Russian Federation
Russian Federation
References
1. Arseev S., Konushin A., Liutov V. Human Recognition by Appearance and Gait. Programming and Computer Software, vol. 44, issue 4, 2018, pp. 258–265
2. Khalid Bashir, Tao Xiang, Shaogang Gong. Gait recognition using gait entropy image. In Proc. of the 3rd international conference on crime detection and prevention, 2009, pp. 1–6.
3. Belhumeur P.N., Hespanha J.P., Kriegman D.J. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, 1997, pp. 711–720.
4. Chen C., Liang J., Zhao H., Hu H., Tian J. Frame difference energy image for gait recognition with incomplete silhouettes. Pattern Recognition Letters, vol. 30, no. 11, 2009, pp 977–984.
5. Francisco Manuel Castro, Manuel J. Marín-Jiménez, Nicolás Guil, Nicolás Pérez de la Blanca. Automatic learning of gait signatures for people identification. Lecture Notes in Computer Science, vol. 10306, 2017. pp. 257–270.
6. Francisco M. Castro, Manuel J. Marín-Jiménez, Rafael Medina Carnicer. Pyramidal Fisher Motion for multiview gait recognition. In Proc. of the 22nd International Conference on Pattern Recognition, 2014, pp. 1692–1697.
7. Dalal N., Triggs B. Histograms of oriented gradients for human detection. In Proc. of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 01. 2005. pp. 886–893.
8. Deng M., Wang C., Cheng F., Zeng W. Fusion of spatial-temporal and kinematic features for gait recognition with deterministic learning. Pattern Recognition, 2017, 67, pp. 186 – 200
9. Feng Y., Li Y., Luo J. Learning effective gait features using LSTM. In Proc. of the 23rd International Conference on Pattern Recognition (ICPR), 2016, pp. 325–330.
10. He Y., Zhang J., Shan H., Wang L. Multitask gans for view-specific feature learning in gait recognition. IEEE Transactions on Information Forensics and Security, vol. 14, no. 1, 2019, pp. 102–113.
11. Han J., Bhanu B. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, issue 2, 2006, pp. 316–322.
12. Hofmann M., Geiger J., Bachmann S., Schuller B., Rigoll G. The TUM Gait from Audio, Image and Depth (GAID) database: Multimodal recognition of subjects and traits. Journal of Visual Communication and Image Representation, vol. 25, no. 1, 2014, pp.195 – 206.
13. Iwama H., Okumura M., Makihara Y., Yagi Y. The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition. IEEE Transactions on Information Forensics and Security, vol. 7, issue 5, 2012, pp.1511–1521.
14. Laptev I., Marszalek M., Schmid C., Rozenfeld B. Learning Realistic Human Actions from Movies. In Proc. of the CVPR 2008 – IEEE Conference on Computer Vision & Pattern Recognition, 2008, pp. 1–8
15. Li C., Sun S., Chen X., Min X. Cross-view gait recognition using joint Bayesian. In Proc. of the Ninth International Conference on Digital Image Processing (ICDIP 2017), 2017.
16. Liu Y., Zhang J., Wang C., Wang L. Multiple HOG templates for gait recognition. In Proc. of the 21st International Conference on Pattern Recognition (ICPR2012). 2012. pp. 2930–2933
17. Makihara Y., Sagawa R., Mukaigawa Y., Echigo T., Yagi Y. Gait recognition using a view transformation model in the frequency domain. Lecture Notes in Computer Science, vol. 3953, 2006, pp. 151–163.
18. Makihara Y., Suzuki A., Muramatsu D., Li X., Yagi Y. Joint intensity and spatial metric learning for robust gait recognition. In Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 6786–6796
19. Mansur A., Makihara Y., Muramatsu D., Yagi Y. Cross-view gait recognition using view-dependent discriminative analysis. In Proc. of the 2014 IEEE/IAPR International Joint Conference on Biometrics, 2014.
20. M.J. Marín-Jiménez, F.M. Castro, N. Guil, F. de la Torre, R. Medina-Carnicer. Deep multitask learning for gait-based biometrics. In Proc. of the IEEE International Conference on Image Processing (ICIP), 2017.
21. Muramatsu D., Makihara Y., Yagi Y. View transformation model incorporating quality measures for cross-view gait recognition. IEEE transactions on cybernetics, 2016, vol. 46, issue 7, pp. 1602–1615.
22. Muramatsu D., Makihara Y., Yagi Y. Crossview gait recognition by fusion of multiple transformation consistency measures. IET Biometrics, vol. 4, issue 2, 2015, pp. 62–73.
23. Shiraga K., Makihara Y., Muramatsu D., Echigo T., Yagi Y. GEINet: View-invariant gait recognition using a convolutional neural network. In Proc. of the 2016 International Conference on Biometrics (ICB), 2016, pp. 1–8.
24. Simonyan K., Zisserman A. Two-stream convolutional networks for action recognition in videos. In Proc. of the of the 27th International Conference on Neural Information Processing Systems, vol. 1 of NIPS’14, 2014, pp. 568–576.
25. Sokolova A., Konushin A. Gait recognition based on convolutional neural networks. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. vol. XLII-2/W4, 2017. pp. 207–212.
26. Sokolova A. and Konushin A. Pose-based Deep Gait Recognition. IET Biometrics, 2018
27. Takemura N., Makihara Y., Muramatsu D. On input/output architectures for convolutional neural network-based crossview gait recognition. IEEE Transactions on Circuits and Systems for Video Technology, 2017.
28. Thapar D., Nigam A., Aggarwa, D., Agarwal P. VGR-net: A view invariant gait recognition network. In Proc. of the EEE 4th International Conference on Identity, Security, and Behavior Analysis (ISBA), 2018, pp. 1-8.
29. Tong S., Fu Y., Ling H., Zhang E. Gait identification by joint spatial-temporal feature. Lecture Notes in Computer Science, vol. 10568, 2017, pp. 457–465.
30. Whytock T., Belyaev A., Robertson N.M. Dynamic distance-based shape features for gait recognition. Journal of Mathematical Imaging and Vision, vol. 50, no. 3, 2014, pp. 314–326.
31. Wu Z., Huang Y., Wang L., Wang X., Tan T. A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 2, 2016, pp. 209–226.
32. Yang Y., Tu D., Li G. Gait recognition using flow histogram energy image. In Proc. of the 22nd International Conference on Pattern Recognition, 2014, pp. 444–449
33. Yu S., Chen H., Reyes E. B. G., Poh, N. GaitGAN: Invariant Gait Feature Extraction Using Generative Adversarial Network. In Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp 532–539
34. Yu S., Chen H., Wang Q., Shen L., Huang Y. Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing, vol. 239, 2017, pp. 81 – 93
35. Yu S., Tan D., Tan T. A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition. In Proc. of the 18th International Conference on Pattern Recognition (ICPR), vol. 4, 2006, pp. 441–444
36. Zhang C. Liu W., Ma H., Fu H. Siamese neural network based gait recognition for human identification. In Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2016, pp. 2832-2836.
37. Zhang X., Sun S., Li C., Zhao X., Hu Y. Deepgait: A learning deep convolutional representation for gait recognition. Lecture Notes in Computer Science, vol. 10568, 2017, pp. 447–456.
Review
For citations:
Sokolova A.I., Konushin A.S. Methods of gait recognition in video. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2019;31(1):69-82. (In Russ.) https://doi.org/10.15514/ISPRAS-2019-31(1)-5
Email this article 