Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology
Fundus image retrieval can help ophthalmologists make evidence-based medico-decision by providing similar cases. Its basic task is to learn highly discriminative visual descriptors from image space, in which lesion features are the main differentiating clue. Lesions in fundus images appear small in size, similar in textures, and scatter around vessels, such as microaneurysms and hemorrhages. Hence, although a single small lesion has a saliently visual manifestation, its discriminative information is hard to reserve in the last image descriptors. For fundus images, the optic disc of the left and right eyes are symmetric, and the macular area lies in the central axis from the vertical view. Based on such spatial structure and lesion characteristics, we present a novel deep metric learning framework equipped with mirror attention to enhance the discriminative features of small and scattering lesions and encode them into image descriptors. The mirror attention can give lesions high attention scores by capturing spatial dependency of vertical and horizontal views, especially the relations between lesions and vessels. Based on the mirror attention, we further propose a new fine triplet loss to confine distances of positive pairs by exploiting the learned relevant degrees of positive pairs in a self-supervised manner. The fine triplet loss can help detect the subtle differences of positive pairs to improve the ranking performance of hit items. To demonstrate the effectiveness of improving retrieval performance, we conduct comprehensive experiments on the largest fundus dataset of diabetic retinopathy (DR) detection and achieve the best precision compared to counterparts. The experiments show that our method produces significant performance improvements for fundus image retrieval, especially the ranking quality of DR grades containing microaneurysms and hemorrhages. Our proposed mirror attention can be applied to off-the-shelf backbones and trained efficiently in an end-to-end manner for other medical images to obtain highly discriminative image descriptors.
General Program of National Natural Science Foundation of China ; Guangdong Provincial Department of Education[2020ZDZX3043]
|WOS Research Area|
|WOS Accession No|
Cited Times [WOS]:1
|Document Type||Journal Article|
|Department||Research Institute of Trustworthy Autonomous Systems|
1.School of Computer Science and Technology,Harbin Institute of Technology,Harbin,China
2.Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering,Southern University of Science and Technology,ShenZhen,China
4.Cixi Institute of Biomedical Engineering,Chinese Academy of Sciences,Ningbo,China
5.Cyberspace Institute of Advanced Technology,Guangzhou University,Guangzhou,China
6.Guangdong Armed Police Hospital,Guangzhou,China
7.The University of Hong Kong,Hong Kong,Hong Kong
|Corresponding Author Affilication||Research Institute of Trustworthy Autonomous Systems; Department of Computer Science and Engineering|
Fang，Jiansheng,Zeng，Ming,Zhang，Xiaoqing,et al. Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology[J]. Biomedical Signal Processing and Control,2023,80.
Fang，Jiansheng.,Zeng，Ming.,Zhang，Xiaoqing.,Liu，Hongbo.,Zhao，Yitian.,...&Liu，Jiang.(2023).Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology.Biomedical Signal Processing and Control,80.
Fang，Jiansheng,et al."Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology".Biomedical Signal Processing and Control 80(2023).
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.