ChatGPT matches doctors in suggesting likely diagnoses in the emergency medicine department

The research, which is also published this month in the Annals of Emergency Medicine included anonymized details on 30 patients who were treated at Jeroen Bosch Hospital’s emergency department in 2022. The researchers entered physicians’ notes on patients’ signs, symptoms, and physical examinations into two versions of ChatGPT. They also provided the chatbot with the results of lab tests, such as blood and urine analysis. For each case, they compared the shortlist of likely diagnoses generated by the chatbot to the shortlist made by emergency medicine doctors and to the patient’s correct diagnosis.

They found a large overlap (around 60%) between the shortlists generated by ChatGPT and the doctors. Doctors had the correct diagnosis within their top five likely diagnoses in 87% of the cases, compared to 97% for ChatGPT version 3.5 and 87% for version 4.0.

Reference: ChatGPT and Generating a Differential Diagnosis Early in an Emergency Department Presentation, Annals of Emergency Medicine, DOI: 10.1016/j.annemergmed.2023.08.003