Commercial AI tool fails to provide additional benefits for screening patients with dense breasts

Korea: Findings from a recent study did not show the benefit of artificial intelligence (AI) with respect to screening mammography performed with supplementary breast ultrasound (US) in dense breast patients. 

"Mammography with supplementary ultrasound showed higher specificity, higher accuracy, and lower recall rate versus mammography with AI as well as in comparison to mammography with both ultrasound and AI," the researchers reported in their study published in the American Journal of Roentgenology. 

The study was conducted by Hee Jung Moon, from the Department of Radiology at South Korea’s Wonju Severance Christian Hospital, Yonsei University Wonju College of Medicine, and colleagues. 

This AJR accepted manuscript included 1,325 women (mean age, 53 years) with dense breasts who underwent both screening mammography and supplementary breast US within a 1-month interval from January 2017 to December 2017; prior mammogram and US were available to compare in 91.2% and 91.8%, respectively. Fifteen radiologists (5 staff and 10 fellows) interpreted mammography and US examinations, and clinical reports were used for Moon et al.’s analysis. A commercially available AI algorithm (Lunit INSIGHT, v 1.1.0.0, Seoul, Korea) was used to retrospectively evaluate mammographic examinations for cancer presence.

Then, screening performances were compared among mammography, AI, US, and test combinations, using generalized estimating equations. At least 24 months of imaging stability was required for a benign diagnosis. 

Key findings include:

• Twelve cancers (6 invasive ductal carcinoma; 6 ductal carcinoma in situ) were diagnosed.
• Mammography, standalone AI, and ultrasound showed cancer detection rate (per 1000 patients) of 6.0, 6.8, and 6.0; recall rate of 4.4%, 11.9%, and 9.2%; sensitivity of 96.2%, 88.7%, and 91.3%; specificity of 96.2%, 88.7%, and 91.3%; and accuracy of 95.9%, 88.5%, and 91.1%.
• Mammography with AI, mammography with ultrasound, and mammography with both ultrasound and AI showed CDR of 7.5, 9.1, and 9.1; recall rate of 14.9, 11.7, and 21.4 (all p<.05); sensitivity of 83.3%, 100.0%, and 100.0%; specificity of 85.8%, 89.1%, and 79.4%; and accuracy of 85.7%, 89.2%, and 79.5%.

Ultimately, mammography with AI, mammography with US, and mammography with both ultrasound and AI showed recall rates of 14.9, 11.7, and 21.4 (all p < .05); sensitivity of 83.3%, 100.0%, and 100.0% (all p > .05); specificity of 85.8%, 89.1%, and 79.4% (all p < .05); and accuracy of 85.7%, 89.2%, and 79.5% (all p < .05).

References:- Si Eun Lee, PhD, Jung Hyun Yoon, PhD, Nak-Hoon Son, PhD, Kyunghwa Han, PhD, and Hee Jung Moon, MD DOI 10.2214/AJR.23.29655.