Study finds ChatGPT to be pretty accurate for common allergy myths

Ten myths were selected and entered in ChatGPT 4.0 with the prompt “true or false.” ChatGPT answers were graded by 24 allergists on a Likert scale of 1-4 (1=inaccurate [100% not true], 2=somewhat inaccurate [>30% not true], 3= somewhat accurate [<10% not true], 4=accurate [100% true]). Finally, allergists were asked their likelihood of using ChatGPT for patient education with a Likert scale of 1-5 (1=not likely at all–5=very likely). Overall, ChatGPT had an accurate or somewhat accurate response in 91% of cases and an inaccurate or somewhat inaccurate response in 9%.

Said Dr. Estrada-Mendizabal, “The response to whether a positive allergy test indicates a clinical allergy was the most accurate response (96% accurate), whereas the response to if hypoallergenic animals were better for people with asthma and allergies was the most inaccurate (17% inaccurate and 21% somewhat inaccurate response). Eighteen allergists (75%) were open (likely or very likely) to using ChatGPT in the future for patient education.”