Deep-Learning Model Shows High Accuracy in Detecting Diabetic Macular Edema: Study
Nguyen and colleagues developed and evaluated a deep-learning model for detecting diabetic macular edema (DME) using three-dimensional optical coherence tomography (OCT) scans. In a real-world screening program, the model achieved accuracy, sensitivity, and specificity above 90%. It also flagged uncertain cases for specialist review, supporting its potential role in improving DME screening and referral. The study was published in The New England Journal of Medicine.
To create a diagnostic tool which is robust and provides patient privacy, researchers developed a deep learning (DL) system using federated learning approach by training and externally validating the model using 8,031 3D OCT volume scans collected from 1,958 diabetic individuals in clinical studies in Hong Kong, the US and Singapore. After initial development of the model, the algorithm was prospectively tested in a clinical diabetic retinopathy screening program in Vietnam. For prospective testing, 1,473 OCT volumes were obtained from 753 diabetic individuals.
To adapt the algorithm to different types of hardware in real time, the researchers used an innovative test-time adaptation method along with a dual-threshold uncertainty range. This uncertainty measure automatically highlights borderline cases and routes them to human ophthalmologists for secondary assessment. Outcome measures included accuracy, sensitivity and specificity of DME diagnosis and subclassification as CI-DME or non-CI-DME.
Key findings:
- Prospective real-time clinical evaluation across 1,473 OCT volumes from 753 patients in Vietnam yielded precise quantitative diagnostic performance metrics.
- The deep learning model achieved an overall diagnostic accuracy of 93.70% (95% CI, 91.24%–94.01%), a sensitivity of 91.78% (95% CI, 86.84%–94.36%), and a specificity of 93.06% (95% CI, 91.53%–94.49%) for detecting the presence of diabetic macular edema.
- For differentiating center-involved DME (CI-DME) from non-center-involved DME (non-CI-DME), the algorithm demonstrated an accuracy of 83.75% (95% CI, 78.17%–88.83%), a sensitivity of 85.61% (95% CI, 79.56%–91.17%), and a specificity of 79.31% (95% CI, 68.75%–89.09%).
- Utilizing the dual-threshold uncertainty range, the algorithm successfully flagged 64 prospective OCT scans as clinically uncertain, directing them for expert re-evaluation by an ophthalmologist.
- Comparative statistical analysis confirmed that the artificial intelligence model achieved diagnostic performance equivalent to certified human ophthalmic experts in identifying DME among diabetic individuals.
The prospective validation of this federated deep learning algorithm clearly shows that it has very high diagnostic accuracy and safety in identifying and staging DME from 3D OCT imaging in a real-world setting. The key conclusions of this paper indicate that the application of privacy-preserving AI through test-time adaptation and uncertainty labeling provides a feasible means to implement specialist-quality retinal screening in developing countries. Implementation of such computational algorithms will go a long way in preventing blindness from diabetic microvascular disease.
Reference:
Nguyen, T. X., Jiang, M., Yang, D., Ran, A. R., Tang, Z., Zhang, S., Hu, X., Tran, V. T., Dai, T. B. L., Le, D. T., Tan, N. T., Szeto, S. K. H., Wong, C. Y. K., Hui, V. W. K., Tsang, K., Chan, C. K. M., Yuen, H. K.-L., Chan, V. T. T., Mak, A. C. Y., … Cheung, C. Y. (2024). Advancing Diabetic Macular Edema Detection from 3D Optical Coherence Tomography Scans: Integrating Privacy-Preserving AI and Generalizability Techniques — A Prospective Validation in Vietnam. NEJM AI, 1(9). https://doi.org/10.1056/aioa2400091
