AI may cover up for lack of availability of experienced ophthalmologists in underdeveloped areas: JAMA

Deep learning (DL)-based screening and referral of patients may help to overcome the shortage of experienced ophthalmologists in underdeveloped regions. The applications of DL techniques trained on color fundus images have shown great potential to provide close to expert performance for the automatic detection of retinal diseases, including diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, myopic maculopathy, retinopathy of prematurity, and papilledema. Although these systems have achieved a good diagnostic performance, nearly all of them could detect only 1 disease or a few diseases and lacked a prospective validation in a clinical setting.

In this study, Li Dong et al developed a DL algorithm, Retinal Artificial Intelligence Diagnosis System (RAIDS), to detect 10 retinal diseases simultaneously. They applied and tested RAIDS at 65 screening centers in 19 provinces of China and compared its performance with the results of a reader study including retinal experts. This multicenter, diagnostic study included individuals attending annual routine medical examinations and participants of population-based and community-based studies.

Based on 1,20,002 ocular fundus photographs, the Retinal Artificial Intelligence Diagnosis System (RAIDS) was developed to identify 10 retinal diseases. RAIDS was validated in a prospective collected data set, and the performance between RAIDS and ophthalmologists was compared in the data sets of the population-based Beijing Eye Study and the community-based Kailuan Eye Study. The performance of each classifier included sensitivity, specificity, accuracy, F1 score, and Cohen κ score.

In the prospective validation data set of 2,08,758 images collected from 1,10,784 individuals, RAIDS achieved a sensitivity of 89.8% to detect any of 10 retinal diseases. RAIDS differentiated 10 retinal diseases with accuracies ranging from 95.3% to 99.9%, without marked differences between medical screening centers and geographical regions in China. Compared with retinal specialists, RAIDS achieved a higher sensitivity for detection of any retinal abnormality (RAIDS, 91.7%; certified ophthalmologists, 83.7%; junior retinal specialists, 86.4%; and senior retinal specialists, 88.5%). RAIDS reached a superior or similar diagnostic sensitivity compared with senior retinal specialists in the detection of 7 of 10 retinal diseases (ie, referral diabetic retinopathy, referral possible glaucoma, macular hole, epiretinal macular membrane, hypertensive retinopathy, myelinated fibers, and retinitis pigmentosa). It achieved a performance comparable with the performance by certified ophthalmologists in 2 diseases (ie, age-related macular degeneration and retinal vein occlusion). Compared with ophthalmologists, RAIDS needed 96% to 97% less time for the image assessment.

In this multicenter diagnostic study conducted at public screening centers and hospitals throughout China, RAIDS achieved high diagnostic accuracy and sensitivity in detecting multiple retinal diseases and saved more than 95% of the examination time. Combining RAIDS with the clinical diagnosis by ophthalmologists achieved a similar diagnostic accuracy and reduced the time needed for examination by 75% as compared with an examination based on ophthalmologists alone.

The scarcity of ophthalmologists in rural regions and the lack of experts in ophthalmology are major factors limiting screening projects for the early detection of blinding diseases. Correspondingly, the agreement between the diagnoses made by certified ophthalmologists and by junior or senior retinal specialists was only moderate in this study. DL-based screening and referral systems may overcome these limitations.

In this study, RAIDS could distinguish 10 common retinal diseases with an accuracy ranging from 95.3% to 99.9%, without major differences between regions. The reader study revealed that the diagnostic accuracy of RAIDS was equal to or better than that of ophthalmologists, including experienced retinal specialists. Furthermore, RAIDS reached a superior or noninferior diagnostic sensitivity compared with retinal specialists in 7 of 10 retinal diseases. These data suggest that RAIDS could be used to provide an independent and automated feedback in health care centers and make referral suggestions. It might help eliminate the gap of resource distribution in underdeveloped regions. In the prospective validation study, all retinal images were evaluated by experts of a retinal expert panel, and referral suggestions were given by phone. The results of the prospective validation revealed that RAIDS had a performance comparable with the retinal experts and suggest that a referral mechanism may be established between local hospitals and medical examinations or reading centers equipped with RAIDS.

These findings suggest that RAIDS achieved a high accuracy in detecting multiple retinal disorders and saved examination time for the screening. RAIDS might be used to provide an automated and immediate referral suggestion in screening and primary care clinic settings, particularly in undeveloped areas, to overcome the shortage of medical resources.

Source: Li Dong et al; JAMA Network Open. 2022;5(5):e229960.

doi:10.1001/jamanetworkopen.2022.9960