Retinal Structure Detection in OCTA Image via Voting-based Multi-task Learning

Hao，Jinkui1; Shen，Ting2; Zhu，Xueli3; Liu，Yonghuai4; Behera，Ardhendu4; Zhang，Dan5; Chen，Bang1; Liu，Jiang6; Zhang，Jiong1; Zhao，Yitian1

2022

10.1109/TMI.2022.3202183

IEEE TRANSACTIONS ON MEDICAL IMAGING   Impact Factor & Quartile

0278-0062

1558-254X

Automated detection of retinal structures, such as retinal vessels (RV), the foveal avascular zone (FAZ), and retinal vascular junctions (RVJ), are of great importance for understanding diseases of the eye and clinical decision-making. In this paper, we propose a novel Voting-based Adaptive Feature Fusion multi-task network (VAFF-Net) for joint segmentation, detection, and classification of RV, FAZ, and RVJ in optical coherence tomography angiography (OCTA). A task-specific voting gate module is proposed to adaptively extract and fuse different features for specific tasks at two levels: features at different spatial positions from a single encoder, and features from multiple encoders. In particular, since the complexity of the microvasculature in OCTA images makes simultaneous precise localization and classification of retinal vascular junctions into bifurcation/crossing a challenging task, we specifically design a task head by combining the heatmap regression and grid classification. We take advantage of three different en face angiograms from various retinal layers, rather than following existing methods that use only a single en face.We carry out extensive experiments on three OCTA datasets acquired using different imaging devices, and the results demonstrate that the proposed method performs on the whole better than either the state-of-the-art single-purpose methods or existing multi-task learning solutions. We also demonstrate that our multi-task learning method generalizes across other imaging modalities, such as color fundus photography, and may potentially be used as a general multi-task learning tool. We also construct three datasets for multiple structure detection, and part of these datasets with the source code and evaluation benchmark have been released for public access.

Bifurcation classification detection Diseases Feature extraction Image segmentation multi-task learning Multitasking OCTA retina structures Retinal vessels segmentation Task analysis

EI

English

Others

20223812753903

Adaptive optics ; Aldehydes ; Classification (of information) ; Computer vision ; Decision making ; Image classification ; Image segmentation ; Learning systems ; Ophthalmology ; Optical tomography ; Signal encoding

Medicine and Pharmacology:461.6 ; Information Theory and Signal Processing:716.1 ; Data Processing and Image Processing:723.2 ; Computer Applications:723.5 ; Light/Optics:741.1 ; Vision:741.2 ; Optical Devices and Systems:741.3 ; Organic Compounds:804.1 ; Information Sources and Analysis:903.1 ; Management:912.2

CLINICAL MEDICINE

2-s2.0-85137909881

Scopus

1.Ningbo Institute of Materials Technology and Engineering, Cixi Institute of Biomedical Engineering, Chinese Academy of Sciences, Ningbo, China
2.Department of Ophthalmology, Second Affiliated Hospital of Zhejiang University, China
3.Ningbo First Hospital, Ningbo, China
4.Department of Computer Science, Edge Hill University, Ormskirk, UK
5.School of Cyber Science and Engineering, Ningbo University of Technology, Ningbo, China
6.Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

Hao，Jinkui,Shen，Ting,Zhu，Xueli,et al. Retinal Structure Detection in OCTA Image via Voting-based Multi-task Learning[J]. IEEE TRANSACTIONS ON MEDICAL IMAGING,2022,PP(99):1-1.

Hao，Jinkui.,Shen，Ting.,Zhu，Xueli.,Liu，Yonghuai.,Behera，Ardhendu.,...&Zhao，Yitian.(2022).Retinal Structure Detection in OCTA Image via Voting-based Multi-task Learning.IEEE TRANSACTIONS ON MEDICAL IMAGING,PP(99),1-1.

Hao，Jinkui,et al."Retinal Structure Detection in OCTA Image via Voting-based Multi-task Learning".IEEE TRANSACTIONS ON MEDICAL IMAGING PP.99(2022):1-1.