RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning

Zhou, Zhanping1,2; Zhao, Chenyang3; Qiao, Hui2,4; Wang, Ming3; Guo, Yuchen2; Wang, Qian3; Zhang, Rui3; Wu, Huaiyu5; Dong, Fajin5; Qi, Zhenhong3; Li, Jianchu3; Tian, Xinping3; Zeng, Xiaofeng3; Jiang, Yuxin3; Xu, Feng1,2; Dai, Qionghai2,4; Yang, Meng3

2022-09-09

10.1016/j.patter.2022.100592

Patterns   Impact Factor & Quartile

2666-3899

Multimodal ultrasound has demonstrated its power in the clinical assessment of rheumatoid arthritis (RA). However, for radiologists, it requires strong experience. In this paper, we propose a rheumatoid arthritis knowledge guided (RATING) system that automatically scores the RA activity and generates interpretable features to assist radiologists' decision-making based on deep learning. RATING leverages the complementary advantages of multimodal ultrasound images and solves the limited training data problem with self-supervised pretraining. RATING outperforms all of the existing methods, achieving an accuracy of 86.1% on a prospective test dataset and 85.0% on an external test dataset. A reader study demonstrates that the RATING system improves the average accuracy of 10 radiologists from 41.4% to 64.0%. As an assistive tool, not only can RATING indicate the possible lesions and enhance the diagnostic performance with multimodal ultrasound but it can also enlighten the road to human-machine collaboration in healthcare.

ESCI

English

Others

International Science and Technology Cooperation Programme[2015DFA30440] ; National Natural Science Foundation of China["81421004","62071271","62021002","61727808","61971447","81301268"] ; National Key Technology R&D Program of China["2019YFC0840603","2017YFC0907601","2017YFC0907604","2017YFE0104200","2018YFA0704000"] ; Chinese Academy of Medical Sciences Innovation Fund for Medical Sciences["2020-I2M-CT-B-035","2021-I2M-1-005"] ; Non-profit Central Research Institute Fund of the Chinese Academy of Medical Sciences[2021-PT320-002] ; Beijing Municipal Natural Science Foundation["JQ18023","JQ19015","JQ21012"]

Computer Science

Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Computer Science, Interdisciplinary Applications

WOS:000898561500003

ELSEVIER

Web of Science

1.Tsinghua Univ, Sch Software, Beijing 100084, Peoples R China
2.Tsinghua Univ, Inst Brain & Cognit Sci, Beijing 100084, Peoples R China
3.Chinese Acad Med Sci & Peking Union Med Coll, Dept Ultrasound, State Key Lab Complex Severe & Rare Dis, Peking Union Med Coll Hosp, Beijing 100730, Peoples R China
4.Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China
5.Jinan Univ, Affiliated Hosp 1, Southern Univ Sci & Technol, Shenzhen Peoples Hosp,Dept Ultrasound,Clin Coll 2, Shenzhen 518020, Peoples R China

Zhou, Zhanping,Zhao, Chenyang,Qiao, Hui,et al. RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning[J]. Patterns,2022,3(9).

Zhou, Zhanping.,Zhao, Chenyang.,Qiao, Hui.,Wang, Ming.,Guo, Yuchen.,...&Yang, Meng.(2022).RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning.Patterns,3(9).

Zhou, Zhanping,et al."RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning".Patterns 3.9(2022).