Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study
Deep neural networks (DNNs) have exhibited impressive performance in the diabetic retinopathy (DR) computer-aided diagnosis (CAD) systems. However, DNNs are quite hungry for enormous labeled images. The limited data volume may degrade both the accuracy and interpretability of DNNs seriously. To alleviate this situation, it is significant to excavate valuable prior information. We aim to explore how to utilize ophthalmologist's eye tracking information into an early DR detection system thus to improve the classification accuracy and interpretability. In this paper, ophthalmologists’ gaze maps are firstly collected from their eye movements through eye tracker during DR diagnosis. Then we investigate an eye tracking based early DR detection model based on ophthalmologists’ gaze maps. First, we analysis the effect of the gaze map integrated with the original fundus image based on two image fusion approaches. Second, the weighted gaze map is regarded as a supervised mask to guide the learning of the attention of a DNN model. Additionally, we propose a novel difficulty-aware and class-adaptive gaze map attention learning strategy to enhance the model interpretability. Comparative experiments prove that the weighted gaze map contains more medical knowledge for diagnostic decision. Meanwhile, the attention guidance method via class activate map (CAM) regularization demonstrates its superiority on improving both the accuracy and interpretability of early DR detection model. These investigations indicate that ophthalmologists’ gaze maps, as medical prior knowledge, can contribute to the design of early DR detection model.
First ; Corresponding
General Program of National Natural Science Foundation of China ; National Natural Science Foundation of China ; Guang-dong Provincial Department of Education[2020ZDZX3043] ; Guangdong Provincial Key Laboratory[2020B121201001] ; Shenzhen Natural Science Fund[JCYJ20200109140820699] ; null
|WOS Research Area|
|WOS Accession No|
Cited Times [WOS]:0
|Document Type||Journal Article|
|Department||Department of Computer Science and Engineering|
1.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
2.Research Institute of Trustworthy Autonomous Systems,Southern University of Science and Technology,Shenzhen,518055,China
3.Department of Biomedical Engineering,College of Future Technology,Peking University,Beijing,100091,China
4.Singapore Eye Research Institute,Singapore National Eye Centre,Singapore,169856,Singapore
5.School of Ophthalmology and Optometry,Wenzhou Medical University,Wenzhou,325035,China
6.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation,Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
|First Author Affilication||Department of Computer Science and Engineering|
|Corresponding Author Affilication||Department of Computer Science and Engineering|
|First Author's First Affilication||Department of Computer Science and Engineering|
Jiang，Hongyang,Hou，Yilin,Miao，Hanpei,et al. Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study[J]. Biomedical Signal Processing and Control,2023,84.
Jiang，Hongyang.,Hou，Yilin.,Miao，Hanpei.,Ye，Haili.,Gao，Mengdi.,...&Liu，Jiang.(2023).Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study.Biomedical Signal Processing and Control,84.
Jiang，Hongyang,et al."Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study".Biomedical Signal Processing and Control 84(2023).
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.