Evaluation of the efficacy of ChatGPT versus medical students in clinical case resolution
DOI:
https://doi.org/10.56294/dm2024.433Keywords:
ChatGPT, Medical education, Clinical case resolution, Artificial intelligence, Student performanceAbstract
Introduction: The use of artificial intelligence (AI) in medical education has gained relevance, and tools like ChatGPT offer support in solving clinical cases. This study compared the average performance of ChatGPT against medical students to evaluate its potential as an educational tool.
Methods: A cross-sectional quantitative study was conducted with 110 sixth-semester medical students from the Technical University of Ambato. Four clinical cases were designed, covering cardiology, endocrinology, gastroenterology, and neurology scenarios. Multiple-choice questions were used to assess both the participants and ChatGPT. Data were analyzed using the Student's t-test for independent samples.
Results: ChatGPT outperformed the students in all cases, with an average score of 8.25 compared to 7.35 for the students. A statistically significant difference was found between the two groups (p = 0.0293).
Conclusions: ChatGPT demonstrated superior performance in solving clinical cases compared to medical students. However, limitations such as potential inaccuracies in information highlight the need for further studies and supervision when integrating AI into medical education.
References
1. Tan S, Xin X, Wu D. ChatGPT in medicine: prospects and challenges: a review article. Int J Surg2024;110(6):3701–6.
2. Hernández R, Moreno SM. El aprendizaje basado en problemas: una propuesta de cualificación docente. Praxis & Saber 2021;12(31):e11174.
3. Rosa N, Palomino J, Cesar U, Piura V, Diaz Espinoza M, Giovanna L, et al. Evaluación del Impacto del aprendizaje basado en proyectos frente a la clase invertida en el desarrollo de habilidades de investigación Comparison of the development of investigative skills between Project Based Learning and the Flipped Class. Ciencia y Tecnología, 2024;28:40. Available from: https://doi.org/10.47460/uct.v28i124.800
4. Valverde-Gutiérrez KV, Esteves-Fajardo ZI. Aprendizaje Basado en Problemas para el Desarrollo del Pensamiento Crítico desde Tempranas Edades. Revista Arbitrada Interdisciplinaria Koinonía 2023;8(1):150–71.
5. Javaid M, Haleem A, Singh RP, Khan S, Khan IH. Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations 2023;3(2).
6. Ruksakulpiwat S, Kumar A, Ajibade A. Using ChatGPT in Medical Research: Current Status and Future Directions. J Multidiscip Healthc2023;16:1513–20.
7. Kim JK, Chua M, Rickard M, Lorenzo A. ChatGPT and large language model (LLM) chatbots: The current state of acceptability and a proposal for guidelines on utilization in academic medicine. J Pediatr Urol (Internet) 2023;19(5):598–604. Available from: https://www.sciencedirect.com/science/article/pii/S1477513123002243
8. Baumgartner C. The opportunities and pitfalls of ChatGPT in clinical and translational medicine. Clin Transl Med 2023;13(3).
9. Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios. J Med Syst 2023;47(1).
10. Goodman RS, Patrinely JR, Stone CA, Zimmerman E, Donald RR, Chang SS, et al. Accuracy and Reliability of Chatbot Responses to Physician Questions. JAMA Netw Open 2023;E2336483.
11. Zampatti S, Peconi C, Megalizzi D, Calvino G, Trastulli G, Cascella R, et al. Innovations in Medicine: Exploring ChatGPT’s Impact on Rare Disorder Management. Genes (Basel)2024;15(4).
12. Dave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell2023;6.
13. Chen TC, Multala E, Kearns P, Delashaw J, Dumont A, Maraganore D, et al. Assessment of ChatGPT’s performance on neurology written board examination questions. BMJ Neurol Open 2023;5(2).
Published
Issue
Section
License
Copyright (c) 2024 Alberto Bustillos , Fernanda Marizande , Andrea Cevallos , Diana Bustillos, Cristina Arteaga , Fabricio Vásquez de la Bandera (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
