How relevant is new version of chatbot for healthcare and research?

ChatGPT is a new AI-based chatbot now being used in medical science. In a recent study, Raju Vaishya from Indraprastha Apollo Hospitals in Sarita Vihar, New Delhi, India, and colleagues aimed to explore the current position and the accuracy of the currently available version of ChatGPT concerning healthcare and medical research.

The study found that the current version of ChatGPT may be helpful for medical personnel in a limited manner as a narrative artificial intelligence chatbot. "However, researchers are advised to fact check all the provided statement, considering its limitations," the authors wrote in their study published in Diabetes & Metabolic Syndrome: Clinical Research & Reviews.

Speaking to Medical Dialogues, Dr Raju Vaishya consultant orthopaedic surgeon IP Apollo hospital, New Delhi and main author of the study said- There have been advancements in the artificial intelligence (AI) and deep learning (DL) in the recent past. A new chatbot, called ChatGPT has become a talk of the town, which interacts in a conversational way and its dialogue format makes is user friendly and fast. In this research study we explored the current position and the accuracy of currently available version of ChatGPT in relation to healthcare and medical research. We found that the current version of ChatGPT generates answers rapidly but narrates data from existing internet literature in a general manner. We noticed errors in responses to medical questions. Hence, we conclude that it may be useful in a limited manner as a narrative AI chatbot for medical personnel, however, researchers are advised to fact check all statements provided, keeping in mind its limitations.

According to Dr Anoop Misra consultant physician and diabetologist at Fortis C DOC hospital and co author of the study- The current version of ChatGPT is just about fine as a narrative tool to smoothen language and seek broad answers to simple scientific questions, but inadequate in assisting many complex science queries.

In addition, they interacted with ChatGPT multiple times to review the accuracy of responses to various medical questions. For this purpose, they searched the online databases from 15th to 25th February 2023 using specific keywords. They found nine results in the Scopus database and 29 results in PubMed.

The study led to the following findings:

• The authors using literature search and interactions with ChatGPT with medical questions, inferred that this version generates rapid answers but narrates data from existing internet literature in a general manner.
• As emphasised by the company (OpenAI Inc.) on the landing page of ChatGPT, the authors found errors in responses to medical questions; further, narrated data were limited up to September 2021.
• Positive features include admitting its limitations in the medical field and learning from previous answers as designed.

To conclude, the current version of ChatGPT appears to be helpful for medical personnel in general. The researchers need to cross-check the correctness of the contents.

"For medical personnel, the ChatGPT's current version may be helpful in a limited manner as a narrative AI chatbot," the investigators wrote, "however, researchers are advised to fact-check all provided statements, keeping in mind its limitations."

Reference:

Vaishya, R., Misra, A., & Vaish, A. (2023). ChatGPT: Is this version good for healthcare and research? Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 17(4), 102744. https://doi.org/10.1016/j.dsx.2023.102744