中文版 | English
Title

基于深度学习的跨场景视频跌倒检测算法研究

Alternative Title
RESEARCH ON VIDEO-BASED FALL DETECTION ALGORITHM FOR CROSS-SCENE BASED ON DEEP LEARNING
Author
Name pinyin
CHEN Qingnan
School number
12032206
Degree
硕士
Discipline
080902 电路与系统
Subject category of dissertation
08 工学
Supervisor
何志海
Mentor unit
电子与电气工程系
Publication Years
2023-05-18
Submission date
2023-07-12
University
南方科技大学
Place of Publication
深圳
Abstract

随着全球老龄化问题日益严峻,老年人跌倒检测成为全社会亟需攻克的关键 领域。基于视频图像的跌倒检测系统存在侵犯老人隐私的问题,但是视频图像是 让相关人员在老年人就医前确认其是否发生跌倒的最有效方式,因此具有重要的 实际应用价值。如何利用深度学习技术便捷地识别视频图像中各种变化场景的跌 倒动作并及时报警,是当今视频跌倒检测的关键问题。现实中老年人的视频跌倒 数据都无法正常获得,公开的跌倒数据集都是研究人员模拟的跌倒,加上相机的位 置、帧率等原因,很容易造成训练场景数据和目标场景测试数据的巨大域分布差 异。通过对深度学习模型泛化问题的深入研究,重点分析了其中的域泛化和无监督 域自适应方法,本文分别提出了一种跨场景泛化的视频跌倒检测算法和两种跨场 景自适应的视频跌倒检测算法,在四个视频跌倒数据集 Le2Fall、URFD、NTU-fall 和 FSD 上,通过一系列实验结果表明,与原场景模型直接泛化使用相比,提出的 跨场景泛化模型或自适应模型均能显著提高目标场景跌倒检测性能,对于实际应 用具有重要意义。同时,针对视频数据集和人体关键点数据集长度不一的问题,本 文提出了用网络主动学习的方法从视频数据集自动生成人体关键点数据集,对于 基于人体关键点的跌倒检测具有重要价值。另外,本文通过结合 AlphaPose 框架和 LSTM、ST-GCN 网络,实现了实时的视频跌倒检测。

Keywords
Language
Chinese
Training classes
独立培养
Enrollment Year
2020
Year of Degree Awarded
2023-06
References List

[1] OF ECONOMIC U N D, SOCIAL AFFAIRS P D. World population prospects 2022: Summaryof results[M]. United Nations New York, NY, USA, 2022: 20.
[2] 童玉芬. 中国人口的最新动态与趋势——结合第七次全国人口普查数据的分析[J]. 中国劳动关系学院学报, 2021: 15-25.
[3] OF ECONOMIC U N D, SOCIAL AFFAIRS P D. World Population Ageing 2020 Highlights:Living arrangements of older persons[EB/OL]. 2021
[2021-11-4]. https://www.un.org/development/desa/pd/news/world-population-ageing-2020-highlights.
[4] BURNS E, KAKARA R. Deaths from falls among persons aged≥ 65 years—United States,2007–2016[J]. Morbidity and Mortality Weekly Report, 2018, 67(18): 509.
[5] 师昉, 李福亮, 张思佳, 等. 中国老年跌倒研究的现状与对策[J]. 中国康复, 2018, 33(3):246-248.
[6] WANG P, CHEN C S, CHUAN C C. Location-aware fall detection system for dementia care onnursing service in evergreen inn of Jianan hospital[C]//2016 IEEE 16th International Conferenceon Bioinformatics and Bioengineering (BIBE). 2016: 309-315.
[7] PIERLEONI P, BELLI A, PALMA L, et al. A high reliability wearable device for elderly falldetection[J]. IEEE Sensors Journal, 2015, 15(8): 4544-4553.
[8] KUMARI M, GUPTA M. An Overview on Wearable Devices for Medical Applications[J].Internet of Medical Things, 2021: 37-50.
[9] SILVA DE LIMA A L, EVERS L J, HAHN T, et al. Freezing of gait and fall detection inParkinson’s disease using wearable sensors: a systematic review[J]. Journal of neurology,2017, 264: 1642-1654.
[10] YU X. Approaches and principles of fall detection for elderly and patient[C]//HealthCom 2008-10th International Conference on e-health Networking, Applications and Services. IEEE, 2008:42-47.
[11] PERRY J T, KELLOG S, VAIDYA S M, et al. Survey and evaluation of real-time fall detection approaches[C]//2009 6th International Symposium on High Capacity Optical Networks andEnabling Technologies (HONET). IEEE, 2009: 158-164.
[12] CHANDER H, BURCH R F, TALEGAONKAR P, et al. Wearable stretch sensors for humanmovement monitoring and fall detection in ergonomics[J]. International journal of environmental research and public health, 2020, 17(10): 3554.
[13] NGUYEN T T, CHO M C, LEE T S. Automatic fall detection using wearable biomedicalsignal measurement terminal[C]//2009 annual international conference of the ieee engineeringin medicine and biology society. IEEE, 2009: 5203-5206.
[14] MUSCI M, DE MARTINI D, BLAGO N, et al. Online fall detection using recurrent neuralnetworks on smart wearable devices[J]. IEEE Transactions on Emerging Topics in Computing,2020, 9(3): 1276-1289.
[15] 石欣, 张涛. 一种可穿戴式跌倒检测装置设计[J]. 仪器仪表学报, 2012, 33(3): 575-580.
[16] WAHEED M, AFZAL H, MEHMOOD K. NT-FDS—A noise tolerant fall detection systemusing deep learning on wearable devices[J]. Sensors, 2021, 21(6): 2006.
[17] LI Y, HO K, POPESCU M. A microphone array system for automatic fall detection[J]. IEEETransactions on Biomedical Engineering, 2012, 59(5): 1291-1301.
[18] PALIPANA S, ROJAS D, AGRAWAL P, et al. FallDeFi: Ubiquitous fall detection using commodity Wi-Fi devices[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2018, 1(4): 1-25.
[19] WANG S, ZHANG X. An approach for fall detection of older population based on multi-sensordata fusion[C]//2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI). IEEE, 2015: 320-323.
[20] TOREYIN B U, SOYER A B, ONARAN I, et al. Falling person detection using multi-sensorsignal processing[J]. EURASIP Journal on Advances in Signal Processing, 2007, 2008: 1-7.
[21] DAHER M, DIAB A, EL NAJJAR M E B, et al. Elder tracking and fall detection system usingsmart tiles[J]. IEEE Sensors Journal, 2016, 17(2): 469-479.
[22] 吕艳, 张萌, 姜吴昊, 等. 采用卷积神经网络的老年人跌倒检测系统设计[J]. 浙江大学学报(工学版), 2019, 53(6): 1130-1138.
[23] HORNG G J, CHEN K H. The smart fall detection mechanism for healthcare under free-livingconditions[J]. Wireless Personal Communications, 2021, 118: 715-753.
[24] TARAMASCO C, RODENAS T, MARTINEZ F, et al. A novel monitoring system for falldetection in older people[J]. Ieee Access, 2018, 6: 43563-43574.
[25] LIU C L, LEE C H, LIN P M. A fall detection system using k-nearest neighbor classifier[J].Expert systems with applications, 2010, 37(10): 7174-7181.
[26] DHOLE S R, KASHYAP A, DANGWAL A N, et al. A novel helmet design and implementationfor drowsiness and fall detection of workers on-site using EEG and Random-Forest Classifier[J]. Procedia Computer Science, 2019, 151: 947-952.
[27] YU M, GONG L, KOLLIAS S. Computer vision based fall detection by a convolutional neuralnetwork[C]//Proceedings of the 19th ACM international conference on multimodal interaction.2017: 416-420.
[28] LI S, XIONG H, DIAO X. Pre-impact fall detection using 3D convolutional neural network[C]//2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). IEEE, 2019:1173-1178.
[29] KIM D E, JEON B, KWON D S. 3D convolutional neural networks based fall detection withthermal camera[J]. The Journal of Korea Robotics Society, 2018, 13(1): 45-54.
[30] YUAN Z, HU H. A fall detection method based on two-stream convolutional neural network[J]. Journal of HeNan Normal University (Natural Science Edition), 2017, 45(3): 102-107.
[31] PENG Y, ZHAO Y, ZHANG J. Two-stream collaborative learning with spatial-temporal attention for video classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(3): 773-786.
[32] CARREIRA J, ZISSERMAN A. Quo vadis, action recognition? a new model and the kineticsdataset[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017: 6299-6308.
[33] KUEHNE H, JHUANG H, GARROTE E, et al. HMDB: a large video database for human motionrecognition[C]//Proceedings of the IEEE International Conference on Computer Vision. IEEE,2011: 2556-2563.
[34] CHARFI I, MITERAN J, DUBOIS J, et al. Definition and performance evaluation of a robustSVM based fall detection solution[C]//2012 eighth international conference on signal imagetechnology and internet based systems. IEEE, 2012: 218-224.
[35] KWOLEK B, KEPSKI M. Human fall detection on embedded platform using depth maps andwireless accelerometer[J]. Computer methods and programs in biomedicine, 2014, 117(3):489-501.
[36] SHAHROUDY A, LIU J, NG T T, et al. Ntu rgb+ d: A large scale dataset for 3d human activityanalysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016: 1010-1019.
[37] BALDEWIJNS G, DEBARD G, MERTES G, et al. Bridging the gap between real-life dataand simulated data by providing a highly realistic fall dataset for evaluating camera-based falldetection algorithms[J]. Healthcare technology letters, 2016, 3(1): 6-11.
[38] SU C, LI J, ZHANG S, et al. Pose-driven deep convolutional model for person re-identification[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 3960-3969.
[39] ZHAO H, TIAN M, SUN S, et al. Spindle net: Person re-identification with human body regionguided feature decomposition and fusion[C]//Proceedings of the IEEE conference on computervision and pattern recognition. 2017: 1077-1085.
[40] NIE X, FENG J, YAN S. Mutual learning to adapt for joint human parsing and pose estimation[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 502-517.
[41] ZHANG Z, SU C, ZHENG L, et al. Correlating edge, pose with parsing[C]//Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 8900-8909.
[42] CHÉRON G, LAPTEV I, SCHMID C. P-cnn: Pose-based cnn features for action recognition[C]//Proceedings of the IEEE International Conference on Computer Vision. 2015: 3218-3226.
[43] DU W, WANG Y, QIAO Y. Rpan: An end-to-end recurrent pose-attention network for action recognition in videos[C]//Proceedings of the IEEE International Conference on ComputerVision. 2017: 3725-3734.
[44] FANG H S, CAO J, TAI Y W, et al. Pairwise body-part attention for recognizing human-objectinteractions[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018:51-67.
[45] LI Y L, ZHOU S, HUANG X, et al. Transferable interactiveness knowledge for human-objectinteraction detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision andPattern Recognition. 2019: 3585-3594.
[46] MICILOTTA A S, ONG E J, BOWDEN R. Real-time upper body detection and 3D pose estimation in monoscopic images[C]//Proceedings of the European Conference on Computer Vision(ECCV). 2006: 139-150.
[47] REN S, HE K, GIRSHICK R, et al. Faster r-cnn: Towards real-time object detection with regionproposal networks[J]. Advances in neural information processing systems, 2015, 28: 1-9.
[48] TOSHEV A, SZEGEDY C. Deeppose: Human pose estimation via deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014: 1653-1660.
[49] TOMPSON J, GOROSHIN R, JAIN A, et al. Efficient object localization using convolutionalnetworks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 648-656.
[50] WEI S E, RAMAKRISHNA V, KANADE T, et al. Convolutional pose machines[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 4724-4732.
[51] FANG H S, LI J, TANG H, et al. AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time[J]. IEEE Transactions on Pattern Analysis and MachineIntelligence, 2022: 1-17.
[52] FANG H S, XIE S, TAI Y W, et al. Rmpe: Regional multi-person pose estimation[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 2334-2343.
[53] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation,1997, 9(8): 1735-1780.
[54] KIPF T N, WELLING M. Semi-Supervised Classification with Graph Convolutional Networks[C]//International Conference on Learning Representations. 2017: 1-14.
[55] YAN S, XIONG Y, LIN D. Spatial temporal graph convolutional networks for skeletonbased action recognition[C]//Proceedings of the AAAI conference on artificial intelligence: volume 32. 2018.
[56] NIEPERT M, AHMED M, KUTZKOV K. Learning convolutional neural networks for graphs[C]//International conference on machine learning. 2016: 2014-2023.
[57] GOWDA S N, ROHRBACH M, SEVILLA-LARA L. Smart frame selection for action recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence: volume 35. 2021:1451-1459.
[58] ZHU W, HU J, SUN G, et al. A key volume mining deep framework for action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016:1991-1999.
[59] MAHASSENI B, LAM M, TODOROVIC S. Unsupervised video summarization with adversarial lstm networks[C]//Proceedings of the IEEE Conference on Computer Vision and PatternRecognition. 2017: 202-211.
[60] MORENO-TORRES J G, RAEDER T, ALAIZ-RODRÍGUEZ R, et al. A unifying view ondataset shift in classification[J]. Pattern recognition, 2012, 45(1): 521-530.
[61] HENDRYCKS D, DIETTERICH T. Benchmarking neural network robustness to common corruptions and perturbations[C]//International Conference on Learning Representations. 2019:1-16.
[62] YANG J, ZHOU K, LI Y, et al. Generalized out-of-distribution detection: A survey[A]. 2021:1-22.
[63] RECHT B, ROELOFS R, SCHMIDT L, et al. Do imagenet classifiers generalize to imagenet?[C]//International conference on machine learning. 2019: 5389-5400.
[64] DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248-255.
[65] LI D, YANG Y, SONG Y Z, et al. Deeper, broader and artier domain generalization[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 5542-5550.
[66] SUYKENS J A, VANDEWALLE J. Least squares support vector machine classifiers[J]. Neuralprocessing letters, 1999, 9: 293-300.
[67] YAO X, BAI Y, ZHANG X, et al. PCL: Proxy-based Contrastive Learning for Domain Generalization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 7097-7107.
[68] CHEN T, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning ofvisual representations[C]//International conference on machine learning. 2020: 1597-1607.
[69] WANG M, DENG W. Deep visual domain adaptation: A survey[J]. Neurocomputing, 2018,312: 135-153.
[70] GUAN H, LIU M. Domain adaptation for medical image analysis: a survey[J]. IEEE Transactions on Biomedical Engineering, 2021, 69(3): 1173-1185.
[71] GANIN Y, LEMPITSKY V. Unsupervised domain adaptation by backpropagation[C]//International conference on machine learning. 2015: 1180-1189.
[72] FANG Y, WANG M, POTTER G G, et al. Unsupervised cross-domain functional MRI adaptation for automated major depressive disorder identification[J]. Medical Image Analysis, 2023,84: 102707.
[73] DONG J, CONG Y, SUN G, et al. Where and How to Transfer: Knowledge AggregationInduced Transferability Perception for Unsupervised Domain Adaptation[J]. IEEE Transactionson Pattern Analysis and Machine Intelligence, 2021, 14(8): 1-17.
[74] SAITO K, WATANABE K, USHIKU Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 3723-3732.
[75] NA J, JUNG H, CHANG H J, et al. Fixbi: Bridging domain spaces for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition. 2021: 1094-1103.
[76] WU Y, INKPEN D, EL-ROBY A. Dual mixup regularized learning for adversarial domainadaptation[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. Springer, 2020: 540-555.
[77] XU M, ZHANG J, NI B, et al. Adversarial domain adaptation with domain mixup[C]//Proceedings of the AAAI conference on artificial intelligence: volume 34. 2020: 6502-6509.
[78] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. The journal of machine learning research, 2016, 17(1): 2096-2030
[79] LIANG J, HU D, FENG J. Do we really need to access the source data? source hypothesistransfer for unsupervised domain adaptation[C]//International Conference on Machine Learning. 2020: 6028-6039.
[80] LIANG J, HU D, WANG Y, et al. Source data-absent unsupervised domain adaptation throughhypothesis transfer and labeling transfer[J]. IEEE Transactions on Pattern Analysis and MachineIntelligence, 2021, 44(11): 8602-8617.
[81] MÜLLER R, KORNBLITH S, HINTON G E. When does label smoothing help?[J]. Advancesin neural information processing systems, 2019, 32: 1-10.
[82] KRAUSE A, PERONA P, GOMES R. Discriminative clustering by regularized informationmaximization[J]. Advances in neural information processing systems, 2010, 23: 1-9.
[83] SHI Y, SHA F. Information-Theoretical Learning of Discriminative Clusters for UnsupervisedDomain Adaptation[C]//International Conference on Machine Learning. 2012: 1275–1282.
[84] HU W, MIYATO T, TOKUI S, et al. Learning discrete representations via information maximizing self-augmented training[C]//International conference on machine learning. 2017: 1558-1567.
[85] CARON M, BOJANOWSKI P, JOULIN A, et al. Deep clustering for unsupervised learning ofvisual features[C]//Proceedings of the European conference on computer vision (ECCV). 2018:132-149.
[86] WALLACE B, HARIHARAN B. Extending and analyzing self-supervised learning across domains[C]//Proceedings of the European conference on computer vision (ECCV). Springer,2020: 717-734.
[87] LIANG J, HE R, SUN Z, et al. Distant supervised centroid shift: A simple and efficient approachto visual domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Visionand Pattern Recognition. 2019: 2975-2984

Academic Degree Assessment Sub committee
电子科学与技术
Domestic book classification number
TP39
Data Source
人工提交
Document TypeThesis
Identifierhttp://kc.sustech.edu.cn/handle/2SGJ60CL/545331
DepartmentDepartment of Electrical and Electronic Engineering
Recommended Citation
GB/T 7714
陈庆南. 基于深度学习的跨场景视频跌倒检测算法研究[D]. 深圳. 南方科技大学,2023.
Files in This Item:
File Name/Size DocType Version Access License
12032206-陈庆南-电子与电气工程(5976KB) Restricted Access--Fulltext Requests
Related Services
Fulltext link
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Export to Excel
Export to Csv
Altmetrics Score
Google Scholar
Similar articles in Google Scholar
[陈庆南]'s Articles
Baidu Scholar
Similar articles in Baidu Scholar
[陈庆南]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[陈庆南]'s Articles
Terms of Use
No data!
Social Bookmark/Share
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.