To evaluate the efficiency of ChatGPT in medical education: an analysis of mcq-based learning and assessment | [Annals of Abbasi Shaheed Hospital & Karachi Medical & Dental College • 2023]

Author(s):

1. Syed Shirjeel Husain: Department of Internal Medicine, Liaquat College of Medicine and Dentistry & Darul Sehat Hospital, Pakistan

2. Zaid Ansari: Department of Internal Medicine, Liaquat College of Medicine and Dentistry & Darul Sehat Hospital, Pakistan

3. Azhar Hussain: Department of Internal Medicine, Liaquat College of Medicine and Dentistry & Darul Sehat Hospital, Pakistan

4. Sahar Zubair Abbasi: Directorate of Medical Education, College of Physicians & Surgeons Pakistan

5. Tahera Ayoob: Department of Oral Surgery, Liaquat College of Medicine and Dentistry & Qamar Dental Hospital, Pakistan

6. Rimsha Mujahid: Department of Internal Medicine, Liaquat College of Medicine and Dentistry & Darul Sehat Hospital, Pakistan

Abstract:

Objective: This study aimed to evaluate the potential of ChatGPT to help students for their assessments via MCQ at different level of cognition by using different subjects of Internal medicine. Methods: This cross-sectional study was conducted in the Department of Internal Medicine in collaboration with post graduate medical education department from June 2023 to August 2023. An MCQ bank was established from three books of MCQ's on subject of Internal Medicine. Total 1428 MCQ's were followed for scrutiny and 307 MCQ's were selected for the assigned task. The selected MCQ's were manually entered one by one in a fresh Chat GPT session. The response was noted against the replies given in respective MCQ's book and marked as correct, not correct or partially correct. MCQ's were categorized as per chapters in Internal medicine and as per cognition level of MCQ's i.e. C1, C2 and C3. Data was analyzed on SPSS version 21.00. Results: Chat GPT replied with 199 correct replies while 98 were wrong and 10 were partially correct. Chat GPT scored 64% overall in all categories. At level of cognition, it solved C2 MCQ's by 80 % but scored 69% and 54% in C1 and C3 categories respectively. Chat GPT replied with 80% accuracy for C2 level MCQ's while results were low for C3 category at around 54%. C1 also had low percentage of correct answers standing close to 69.8%. Almost all subjects showed healthy responses around the mean except for endocrinology and hematology where responses are below 60% and 40% respectively. Conclusion: This study findings suggest that ChatGPT is a useful tool for students and medical educationist with its current framework but a subtle approach should be inclined towards its role in future.

Page(s): 194-200

DOI: DOI not available

Published: Journal: Annals of Abbasi Shaheed Hospital & Karachi Medical & Dental College, Volume: 28, Issue: 4, Year: 2023

Keywords:

medical education , Artificial intelligence , Educational assessment

References:

References are not available for this document.

Citations

Citations are not available for this document.

Citations

Downloads

Views