AI Matches Radiologists in Detecting TB and Chest X-Ray Abnormalities, Finds Multi-Site Study

"AI matched the performance of radiologists in detecting active tuberculosis and also showed promise in identifying other chest X-ray abnormalities effectively," the researchers wrote in NEJM AI.

To enhance TB screening in resource-limited settings, researchers explored using artificial intelligence to interpret chest X-rays as a potential triage tool for active pulmonary tuberculosis. Sahar Kazemzadeh, Google, Mountain View, CA, and colleagues aimed to evaluate the effectiveness of a commercially available CXR-based AI system in detecting active TB in real-world clinical settings.

For this purpose, the researchers assessed two cloud-based chest X-ray AI tools—one for detecting tuberculosis and the other for identifying general CXR abnormalities—in a high-TB and HIV burden population. The study included 1,978 adults with TB symptoms, recent TB exposure, or newly diagnosed HIV across three clinical sites.

The TB AI scores were evaluated using two thresholds: one for high sensitivity and another to mirror radiologist performance. Ten radiologists reviewed the X-rays independently, blinded to the reference results. The primary aim was to test whether the TB AI was noninferior to radiologists, while secondary analyses compared its performance with WHO targets. The abnormality AI was also evaluated against triage-appropriate sensitivity and specificity benchmarks.

Key Findings:

• Among the 1,910 patients analyzed, 1,827 (96%) had a conclusive TB diagnosis. Of these, 649 patients (36%) were HIV positive, and 192 patients (11%) were TB positive.
• The TB AI demonstrated 87% sensitivity and 70% specificity at the high-sensitivity threshold.
• At the balanced threshold, the TB AI showed 78% sensitivity and 82% specificity.
• Radiologists had a mean sensitivity of 76% and a mean specificity of 82%.
• At the high-sensitivity threshold, TB AI was noninferior to the average radiologist's sensitivity but not to specificity.
• TB AI exceeded the WHO target for specificity but did not meet the sensitivity benchmark.
• At the balanced threshold, the performance of TB AI was comparable to that of radiologists.
• The abnormality AI achieved 97% sensitivity and 79% specificity, meeting both predefined performance targets.

The study concluded that the chest X-ray-based AI system for tuberculosis detection performed on par with radiologists when used for triaging active pulmonary TB in populations with high TB and HIV prevalence. However, neither the AI tool nor the radiologists met the World Health Organization's recommended sensitivity levels in this setting. The AI system also showed promise in identifying other chest X-ray abnormalities, highlighting its potential for broader clinical use.

Reference:

Kazemzadeh S, Kiraly AP, Nabulsi Z, Sanjase N, Maimbolwa M, Shuma B, Jamshy S, Chen C, Agharwal A, Lau CT, Sellergren A, Golden D, Yu J, Wu E, Matias Y, Chou K, Corrado GS, Shetty S, Tse D, Eswaran K, Liu Y, Pilgrim R, Muyoyeta M, Prabhakara S. Prospective Multi-Site Validation of AI to Detect Tuberculosis and Chest X-Ray Abnormalities. NEJM AI. 2024 Oct;1(10):10.1056/aioa2400018. doi: 10.1056/aioa2400018. Epub 2024 Sep 26. PMID: 39823016; PMCID: PMC11737584.