ChatGPT's Attitude, Knowledge, and Clinical Application in Geriatrics Practice and Education: Exploratory Observational Study.

The increasing use of ChatGPT in clinical practice and medical education necessitates the evaluation of its reliability, particularly in geriatrics. This study aimed to evaluate ChatGPT's trustworthiness in geriatrics through 3 distinct approaches: evaluating ChatGPT's geriatrics attitude, knowledge, and clinical application with 2 vignettes of geriatric syndromes (polypharmacy and falls). We used the validated University of California, Los Angeles, geriatrics attitude and knowledge instruments to evaluate ChatGPT's geriatrics attitude and knowledge and compare its performance with that of medical students, residents, and geriatrics fellows from reported results in the literature. We also evaluated ChatGPT's application to 2 vignettes of geriatric syndromes (polypharmacy and falls). The mean total score on geriatrics attitude of ChatGPT was significantly lower than that of trainees (medical students, internal medicine residents, and geriatric medicine fellows; 2.7 vs 3.7 on a scale from 1-5; 1=strongly disagree; 5=strongly agree). The mean subscore on positive geriatrics attitude of ChatGPT was higher than that of the trainees (medical students, internal medicine residents, and neurologists; 4.1 vs 3.7 on a scale from 1 to 5 where a higher score means a more positive attitude toward older adults). The mean subscore on negative geriatrics attitude of ChatGPT was lower than that of the trainees and neurologists (1.8 vs 2.8 on a scale from 1 to 5 where a lower subscore means a less negative attitude toward aging). On the University of California, Los Angeles geriatrics knowledge test, ChatGPT outperformed all medical students, internal medicine residents, and geriatric medicine fellows from validated studies (14.7 vs 11.3 with a score range of -18 to +18 where +18 means that all questions were answered correctly). Regarding the polypharmacy vignette, ChatGPT not only demonstrated solid knowledge of potentially inappropriate medications but also accurately identified 7 common potentially inappropriate medications and 5 drug-drug and 3 drug-disease interactions. However, ChatGPT missed 5 drug-disease and 1 drug-drug interaction and produced 2 hallucinations. Regarding the fall vignette, ChatGPT answered 3 of 5 pretests correctly and 2 of 5 pretests partially correctly, identified 6 categories of fall risks, followed fall guidelines correctly, listed 6 key physical examinations, and recommended 6 categories of fall prevention methods. This study suggests that ChatGPT can be a valuable supplemental tool in geriatrics, offering reliable information with less age bias, robust geriatrics knowledge, and comprehensive recommendations for managing 2 common geriatric syndromes (polypharmacy and falls) that are consistent with evidence from guidelines, systematic reviews, and other types of studies. ChatGPT's potential as an educational and clinical resource could significantly benefit trainees, health care providers, and laypeople. Further research using GPT-4o, larger geriatrics question sets, and more geriatric syndromes is needed to expand and confirm these findings before adopting ChatGPT widely for geriatrics education and practice.

Cheng HY 《-》

Erratum: Eyestalk Ablation to Increase Ovarian Maturation in Mud Crabs.

《Jove-Journal of Visualized Experiments》

Application of Large Language Models in Medical Training Evaluation-Using ChatGPT as a Standardized Patient: Multimetric Assessment.

With the increasing interest in the application of large language models (LLMs) in the medical field, the feasibility of its potential use as a standardized patient in medical assessment is rarely evaluated. Specifically, we delved into the potential of using ChatGPT, a representative LLM, in transforming medical education by serving as a cost-effective alternative to standardized patients, specifically for history-taking tasks. The study aims to explore ChatGPT's viability and performance as a standardized patient, using prompt engineering to refine its accuracy and use in medical assessments. A 2-phase experiment was conducted. The first phase assessed feasibility by simulating conversations about inflammatory bowel disease (IBD) across 3 quality groups (good, medium, and bad). Responses were categorized based on their relevance and accuracy. Each group consisted of 30 runs, with responses scored to determine whether they were related to the inquiries. For the second phase, we evaluated ChatGPT's performance against specific criteria, focusing on its anthropomorphism, clinical accuracy, and adaptability. Adjustments were made to prompts based on ChatGPT's response shortcomings, with a comparative analysis of ChatGPT's performance between original and revised prompts. A total of 300 runs were conducted and compared against standard reference scores. Finally, the generalizability of the revised prompt was tested using other scripts for another 60 runs, together with the exploration of the impact of the used language on the performance of the chatbot. The feasibility test confirmed ChatGPT's ability to simulate a standardized patient effectively, differentiating among poor, medium, and good medical inquiries with varying degrees of accuracy. Score differences between the poor (74.7, SD 5.44) and medium (82.67, SD 5.30) inquiry groups (P<.001), between the poor and good (85, SD 3.27) inquiry groups (P<.001) were significant at a significance level (α) of .05, while the score differences between the medium and good inquiry groups were not statistically significant (P=.16). The revised prompt significantly improved ChatGPT's realism, clinical accuracy, and adaptability, leading to a marked reduction in scoring discrepancies. The score accuracy of ChatGPT improved 4.926 times compared to unrevised prompts. The score difference percentage drops from 29.83% to 6.06%, with a drop in SD from 0.55 to 0.068. The performance of the chatbot on a separate script is acceptable with an average score difference percentage of 3.21%. Moreover, the performance differences between test groups using various language combinations were found to be insignificant. ChatGPT, as a representative LLM, is a viable tool for simulating standardized patients in medical assessments, with the potential to enhance medical training. By incorporating proper prompts, ChatGPT's scoring accuracy and response realism significantly improved, approaching the feasibility of actual clinical use. Also, the influence of the adopted language is nonsignificant on the outcome of the chatbot.

Wang C ，Li S ，Lin N ，Zhang X ，Han Y ，Wang X ，Liu D ，Tan X ，Pu D ，Li K ，Qian G ，Yin R ... -  《JOURNAL OF MEDICAL INTERNET RESEARCH》

Health profession students' perceptions of ChatGPT in healthcare and education: insights from a mixed-methods study.

The aim of this study was to investigate the perceptions of health profession students regarding ChatGPT use and the potential impact of integrating ChatGPT in healthcare and education. Artificial Intelligence is increasingly utilized in medical education and clinical profession training. However, since its introduction, ChatGPT remains relatively unexplored in terms of health profession students' acceptance of its use in education and practice. This study employed a mixed-methods approach, using a web-based survey. The study involved a convenience sample recruited through various methods, including Faculty of Medicine announcements, social media, and snowball sampling, during the second semester (March to June 2023). Data were collected using a structured questionnaire with closed-ended questions and three open-ended questions. The final sample comprised 217 undergraduate health profession students, including 73 (33.6%) nursing students, 65 (30.0%) medical students, and 79 (36.4%) occupational therapy, physiotherapy, and speech therapy students. Among the surveyed students, 86.2% were familiar with ChatGPT, with generally positive perceptions as reflected by a mean score of 4.04 (SD = 0.62) on a scale of 1 to 5. Positive feedback was particularly noted with respect to ChatGPT's role in information retrieval and summarization. The qualitative data revealed three main themes: experiences with ChatGPT, its impact on the quality of healthcare, and its integration into the curriculum. The findings highlight benefits such as serving as a convenient tool for accessing information, reducing human errors, and fostering innovative learning approaches. However, they also underscore areas of concern, including ethical considerations, challenges in fostering critical thinking, and issues related to verification. The absence of significant differences between the different fields of study indicates consistent perceptions across nursing, medicine, and other health profession students. Our findings underscore the necessity for continuous refinement to enhance ChatGPT's accuracy, reliability, and alignment with the diverse educational needs of health professions. These insights not only deepen our understanding of student perceptions of ChatGPT in healthcare education but also have significant implications for the future integration of AI in health profession practice. The study emphasizes the importance of a careful balance between leveraging the benefits of AI tools and addressing ethical and pedagogical concerns.

Moskovich L ，Rozani V 《BMC Medical Education》

Defining the optimum strategy for identifying adults and children with coeliac disease: systematic review and economic modelling.

Elwenspoek MM ，Thom H ，Sheppard AL ，Keeney E ，O'Donnell R ，Jackson J ，Roadevin C ，Dawson S ，Lane D ，Stubbs J ，Everitt H ，Watson JC ，Hay AD ，Gillett P ，Robins G ，Jones HE ，Mallett S ，Whiting PF ... -  《-》