AI vs. Clinicians: Study Explores the Future of Diagnosing in Healthcare

A study from the University of Maine has compared the diagnostic abilities of artificial intelligence (AI) models with those of human clinicians, shedding light on the strengths and limitations of both. Published in the Journal of Health Organization and Management, the study examined over 7,000 anonymized medical queries from the U.S. and Australia, offering insights into how AI could support but not replace healthcare professionals in a strained system.

The researchers assessed how both AI and clinicians responded to various medical questions, categorizing them into factual, procedural, and emotional or interpretive queries. The results showed that AI performed well on objective tasks, with high alignment to expert standards for factual and procedural information. However, the technology often faltered when addressing complex “why” and “how” questions, particularly those requiring emotional nuance or context.

Inconsistencies also emerged when the same questions were posed across multiple sessions, raising concerns about reliability. “This isn’t about replacing doctors and nurses,” said C. Matt Graham, author of the study and associate professor of information systems and security management at the Maine Business School. “It’s about augmenting their abilities. AI can be a second set of eyes; it can help clinicians sift through mountains of data, recognize patterns and offer evidence-based recommendations in real time.”

Importantly, AI responses often lacked the empathy and human connection vital to effective patient care. “Healthcare professionals offer healing that is grounded in human connection, through sight, touch, presence and communication — experiences that AI cannot replicate,” said Kelley Strout, associate professor of UMaine’s School of Nursing, who was not involved in the study. “The synergy between AI and clinicians’ judgment, compassion and application of evidence-based practice has the potential to transform healthcare systems but only if accompanied by rigorous standards, ethical frameworks and safeguards to monitor for errors and unintended consequences.”

While AI offers potential relief, the authors caution against unchecked deployment, citing the need for oversight, ethical safeguards, and inclusivity in design.

Reference: Graham, C.M. (2025), "Artificial intelligence vs human clinicians: a comparative analysis of complex medical query handling across the USA and Australia", Journal of Health Organization and Management, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JHOM-02-2025-0100