ChatGPT may help residents improve their understanding of orthopaedic surgery and general orthopaedic principles

Publicly available AI language models such as ChatGPT have demonstrated utility in text generation and even problem-solving when provided with clear instructions. Amidst this transformative shift Diane Ghanem et al conducted a study to assess ChatGPT's performance on the orthopaedic surgery in-training examination (OITE). Investigation performed at The Johns Hopkins Hospital, Baltimore, Maryland. It has been published in “JBJS Open Access.”

All 213 OITE 2021 web-based questions were retrieved from the AAOS-ResStudy website (https://www.aaos. org/education/examinations/ResStudy). Two independent reviewers copied and pasted the questions and response options into ChatGPT Plus (version 4.0) and recorded the generated answers. All media-containing questions were flagged and carefully examined. Twelve OITE media-containing questions that relied purely on images (clinical pictures, radio graphs, MRIs, CT scans) and could not be rationalized from the clinical presentation were excluded. Cohen's Kappa coefficient was used to examine the agreement of ChatGPT-generated responses between reviewers. Descriptive statistics were used to summarize the performance (% correct) of ChatGPT Plus. The 2021 norm table was used to compare ChatGPT Plus' performance on the OITE to national orthopaedic surgery residents in that same year.