USA: In a new study, ChatGPT was asked to rate 10 common allergy myths, and it judged their accuracy correctly 91% of the time. The study findings were presented at the American College of Allergy, Asthma and Immunology (ACAAI) Annual Scientific Meeting in Anaheim, California.
Artificial intelligence language models such as ChatGPT are increasingly being used in many different professions, and medicine is not an exception.
“We searched the PubMed and Google Scholar databases to identify all articles on common allergy myths between 2000-2023,” says Ricardo Estrada-Mendizabal, MD, ACAAI member and lead author of the study. “Our goal was to assess the accuracy of ChatGPT in correctly identifying common allergy myths. We know allergy and asthma patients are constantly exposed to misinformation on the internet. We thought that the surge of public language models could represent a bigger threat to misinformation spread, especially because ChatGPT is unable to provide references.”
Ten myths were selected and entered in ChatGPT 4.0 with the prompt “true or false.” ChatGPT answers were graded by 24 allergists on a Likert scale of 1-4 (1=inaccurate [100% not true], 2=somewhat inaccurate [>30% not true], 3= somewhat accurate [<10% not true], 4=accurate [100% true]). Finally, allergists were asked their likelihood of using ChatGPT for patient education with a Likert scale of 1-5 (1=not likely at all–5=very likely). Overall, ChatGPT had an accurate or somewhat accurate response in 91% of cases and an inaccurate or somewhat inaccurate response in 9%.
Said Dr. Estrada-Mendizabal, “The response to whether a positive allergy test indicates a clinical allergy was the most accurate response (96% accurate), whereas the response to if hypoallergenic animals were better for people with asthma and allergies was the most inaccurate (17% inaccurate and 21% somewhat inaccurate response). Eighteen allergists (75%) were open (likely or very likely) to using ChatGPT in the future for patient education.”
Disclaimer: This website is primarily for healthcare professionals. The content here does not replace medical advice and should not be used as medical, diagnostic, endorsement, treatment, or prescription advice. Medical science evolves rapidly, and we strive to keep our information current. If you find any discrepancies, please contact us at corrections@medicaldialogues.in. Read our Correction Policy here. Nothing here should be used as a substitute for medical advice, diagnosis, or treatment. We do not endorse any healthcare advice that contradicts a physician's guidance. Use of this site is subject to our Terms of Use, Privacy Policy, and Advertisement Policy. For more details, read our Full Disclaimer here.
NOTE: Join us in combating medical misinformation. If you encounter a questionable health, medical, or medical education claim, email us at factcheck@medicaldialogues.in for evaluation.