SCORE-AI achieved diagnostic accuracy similar to human experts in EEG diagnosis: JAMA

Electroencephalography (EEG) is most reliable diagnostic tool as it provides essential information to aid diagnosis and classification of epilepsy, important for therapeutic decision-making however requires special expertise unavailable in many regions of the world.

Researchers aimed to develop and validate an AI model (Standardized Computer-based Organized Reporting of EEG–Artificial Intelligence [SCORE-AI]) with the ability to distinguish abnormal from normal EEG recordings and to classify abnormal EEG recordings into categories relevant for clinical decision-making: epileptiform-focal, epileptiform-generalized, nonepileptiform-focal, and nonepileptiform-diffuse.

The diagnostic study published in JAMA Neurology, they found that an AI model (SCORE-AI) separated normal from abnormal recordings then classify abnormal recordings as epileptiform-focal, epileptiform-generalized, nonepileptiform-focal, or nonepileptiform-diffuse with a diagnostic accuracy similar to that of human experts.

The multicenter diagnostic accuracy study, a convolutional neural network model, SCORE-AI, was developed and validated using EEGs recorded between 2014 and 2020. A total of 30 493 recordings of patients referred for EEG were included into the development data set annotated by 17 experts. Patients aged more than 3 months and not critically ill were eligible. The SCORE-AI was validated using 3 independent test data sets: a multicenter data set of 100 representative EEGs evaluated by 11 experts, a single-center data set of 9785 EEGs evaluated by 14 experts, and for benchmarking with previously published AI models, a data set of 60 EEGs with external reference standard.

The key findings of the study are

• The characteristics of the EEG data sets include development data set (N = 30 493; 14 980 men), multicenter test data set (N = 100; 61 men,), single-center test data set (N = 9785; 5168 men) and test data set with external reference standard (N = 60; 27 men).

• The SCORE-AI achieved high accuracy, with an area under the receiver operating characteristic curve between 0.89 and 0.96 for the different categories of EEG abnormalities, and performance similar to human experts.

• Benchmarking against 3 previously published AI models was limited to comparing detection of epileptiform abnormalities.

• The accuracy of SCORE-AI (88.3%; 95% CI, 79.2%-94.9%) was significantly higher than the 3 previously published models (P < .001) and similar to human experts.

Jesper Tveit, PhD and team concluded that “In this study, SCORE-AI achieved human expert level performance in fully automated interpretation of routine EEGs. Application of SCORE-AI may improve diagnosis and patient care in underserved areas and improve efficiency and consistency in specialized epilepsy centers.”

Reference: Tveit J, Aurlien H, Plis S, et al. Automated Interpretation of Clinical Electroencephalograms Using Artificial Intelligence. JAMA Neurol. Published online June 20, 2023. doi:10.1001/jamaneurol.2023.1645.