AI Tool Improves Detection of Intracranial Aneurysms, Suggests study
USA: Researchers have found in a new study that a commercially available AI tool identified 39% more intracranial aneurysms than radiologists alone when evaluated in “shadow mode” at Northwell Health. In this prospective study of 3,856 consecutive CT angiography (CTA) examinations across inpatient, emergency, and outpatient settings, AI performance varied by clinical setting, but overall findings support its potential as a collaborative diagnostic aid for improving aneurysm detection.
The study, published in the Journal of the American College of Radiology by Shlomit Goldberg-Stein, MD, Director of Artificial Intelligence, Department of Radiology, Northwell Health, New Hyde Park, New York, and colleagues, assessed the real-world performance of an FDA-cleared AI algorithm designed to identify intracranial aneurysms on brain CTA.
Small intracranial aneurysms can be challenging to detect. The researchers assessed whether AI could improve aneurysm detection alongside radiologist interpretation without substantially increasing false positives.
The prospective shadow-mode study analyzed 3,856 consecutive brain CTA examinations conducted between November 7 and December 19, 2023. Radiologists were blinded to the AI results, while Aidoc’s AI algorithm independently analyzed the scans. Neuroradiologists reviewed discordant cases.
Of the 3,856 CTAs, 195 (5.1%) were positive for aneurysms, with AI and radiologists agreeing in 3,714 cases (96.3%).
Key findings:
- AI demonstrated higher sensitivity than radiologists alone, at 84.6% versus 71.8%, while specificity was similar at 98.7% and 98.5%, respectively.
- AI identified 55 additional true-positive aneurysms that were not detected by radiologists.
- The relative enhanced detection rate was 39%, indicating a potential 39% increase in aneurysm detection when AI was incorporated.
- The AI-to-radiologist incremental detection ratio was 1.83, while the gain-to-pain ratio was 1.20, suggesting that additional detections outweighed the false-positive burden overall.
- The number needed to examine was 70.1 examinations to identify one additional aneurysm.
- The greatest operational benefit was observed among inpatients, with a 78.3% relative enhanced detection rate, a gain-to-pain ratio of 2.57, and a number needed to examine of 28.9.
- Emergency cases showed a 37.1% relative enhanced detection rate and a gain-to-pain ratio of 1.00.
- Outpatient examinations showed lower operational benefit, with a 14.1% relative enhanced detection rate and a gain-to-pain ratio of 0.67.
- More than half of the aneurysms detected only by AI were smaller than 3 mm, accounting for 31 of 55 cases (56.4%).
The pattern of detection suggested that AI and radiologists may identify different subsets of aneurysms. AI-only detections were predominantly very small lesions, whereas radiologist-only detections were more commonly 3–5 mm aneurysms.
The researchers concluded that the AI tool demonstrated favorable overall operational performance and could serve as a complementary aid to radiologists interpreting brain CTA. However, its clinical value varied according to the care setting, highlighting the importance of evaluating AI performance within specific real-world workflows before widespread implementation.
Reference:
Goldberg-Stein, S., Sanmartin, M. X., Saks, R., Varghese, J., Mir, Y., Gandomi, A., Scheiner, J., Patel, R. D., Hirschorn, D., Rula, E. Y., Naidich, J. J., Barish, M. A., & Sanelli, P. C. (2026). Prospective Shadow-Mode Evaluation of an Artificial Intelligence Tool for Intracranial Aneurysm Detection on CT Angiography: Incremental Yield and Operational Impact. Journal of the American College of Radiology. https://doi.org/10.1016/j.jacr.2026.07.018
Next Story