Can GPT-4 Improve Diagnosis? Study Provides Insights - Video

A new study in JAMA Network Open found that the availability of GPT-4 to physicians as a diagnostic aid did not significantly improve clinical reasoning compared to conventional resources. A collaborative team of researchers from the University of Minnesota Medical School, Stanford University, Beth Israel Deaconess Medical Center and the University of Virginia studied how well doctors used GPT-4 — an

artificial intelligence (AI) large language model system — for diagnosing patients.

The study was conducted with 50 U.S.-licensed physicians in family medicine, internal medicine and emergency medicine. Key findings include:

GPT-4 alone demonstrated significantly better scores in diagnostic performance, surpassing the performance of clinicians using conventional diagnostic online resources and clinicians assisted by GPT-4.

There was no significant enhancement in diagnostic performance with the addition of GPT-4 when assessing clinicians using GPT-4 against clinicians using conventional diagnostic resources.

These results underline the complexity of integrating AI into clinical practice. While GPT-4 alone showed promising results, the integration of GPT-4 as a diagnostic aid alongside clinicians did not significantly outperform the use of conventional diagnostic resources. This suggests a nuanced potential for AI in healthcare, emphasizing the importance of further exploration into how AI can best support clinical practice.

Reference: Goh E, Gallo R, Hom J, et al. Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial. JAMA Netw Open. 2024;7(10):e2440969. doi:10.1001/jamanetworkopen.2024.40969