A Fall Detection Network by 2D/3D Spatio-temporal Joint Models with Tensor Compression on Edge
Falling is ranked highly among the threats in elderly healthcare, which promotes the development of automatic fall detection systems with extensive concern. With the fast development of the Internet of Things (IoT) and Artificial Intelligence (AI), camera vision-based solutions have drawn much attention for single-frame prediction and video understanding on fall detection in the elderly by using Convolutional Neural Network (CNN) and 3D-CNN, respectively. However, these methods hardly supervise the intermediate features with good accurate and efficient performance on edge devices, which makes the system difficult to be applied in practice. This work introduces a fast and lightweight video fall detection network based on a spatio-temporal joint-point model to overcome these hurdles. Instead of detecting fall motion by the traditional CNNs, we propose a Long Short-Term Memory (LSTM) model based on time-series joint- point features extracted from a pose extractor. We also introduce the increasingly mature RGB-D camera and propose 3D pose estimation network to further improve the accuracy of the system. We propose to apply tensor train decomposition on the model to reduce storage and computational consumption so the deployment on edge devices can to realized. Experiments are conducted to verify the proposed framework. For fall detection task, the proposed video fall detection framework achieves a high sensitivity of 98.46% on Multiple Cameras Fall, 100% on UR Fall, and 98.01% on NTU RGB-D 120. For pose estimation task, our 2D model attains 73.3 mAP in the COCO keypoint challenge, which outperforms the OpenPose by 8%. Our 3D model attains 78.6% mAP on NTU RGB-D dataset with 3.6x faster speed than OpenPose.
National Natural Science Foundation of China (NSFC) ; Shenzhen Science and Technology Program[KQTD2020020113051096] ; Innovative Team Program of Education Department of Guangdong Province[2018KCXTD028]
|WOS Research Area|
Computer Science, Hardware & Architecture ; Computer Science, Software Engineering
|WOS Accession No|
Cited Times [WOS]:0
|Document Type||Journal Article|
|Department||SUSTech Institute of Microelectronics|
Southern University of Science and Technology
|First Author Affilication||Southern University of Science and Technology|
|Corresponding Author Affilication||Southern University of Science and Technology|
|First Author's First Affilication||Southern University of Science and Technology|
Shuwei,Li,Changhai,Man,Ao,Shen,et al. A Fall Detection Network by 2D/3D Spatio-temporal Joint Models with Tensor Compression on Edge[J]. ACM Transactions on Embedded Computing Systems,2022,21(6).
Shuwei,Li.,Changhai,Man.,Ao,Shen.,Ziyi,Guan.,Wei,Mao.,...&Hao,Yu.(2022).A Fall Detection Network by 2D/3D Spatio-temporal Joint Models with Tensor Compression on Edge.ACM Transactions on Embedded Computing Systems,21(6).
Shuwei,Li,et al."A Fall Detection Network by 2D/3D Spatio-temporal Joint Models with Tensor Compression on Edge".ACM Transactions on Embedded Computing Systems 21.6(2022).
|Files in This Item:|
|J105.A Fall Detectio（7063KB）||Restricted Access||--|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.