Generative AI's diagnostic capabilities comparable to non-specialist doctors, claims research

Of the large language models (LLMs) that were analyzed, ChatGPT was the most commonly studied.

The comparative evaluation revealed that medical specialists had a 15.8% higher diagnostic accuracy than generative AI. The average diagnostic accuracy of generative AI was 52.1%, with the latest models of generative AI sometimes showing accuracy on par with non-specialist doctors.

"This research shows that generative AI's diagnostic capabilities are comparable to non-specialist doctors. It could be used in medical education to support non-specialist doctors and assist in diagnostics in areas with limited medical resources." stated Dr. Takita.

"Further research, such as evaluations in more complex clinical scenarios, performance evaluations using actual medical records, improving the transparency of AI decision-making, and verification in diverse patient groups, is needed to verify AI's capabilities."

The findings were published in npj Digital Medicine.

Reference:

Hirotaka Takita, Daijiro Kabata, Shannon L. Walston, Hiroyuki Tatekawa, Kenichi Saito, Yasushi Tsujimoto, Yukio Miki, Daiju Ueda. A systematic review and meta-analysis of diagnostic performance comparison between generative AI and physicians. npj Digital Medicine, 2025; 8 (1) DOI: 10.1038/s41746-025-01543-z