Abstract:
Objective: To analyse and compare the assessment and grading of human-written and machine-written formative essays. Study Design: Quasi-experimental, qualitative cross-sectional study. Place and Duration of the Study: Department of Science of Dental Materials, Hamdard College of Medicine & Dentistry, Hamdard University, Karachi, from February to April 2023. Methodology: Ten short formative essays of nal-year dental students were manually assessed and graded. These essays were then graded using ChatGPT version 3.5. The chatbot responses and prompts were recorded and matched with manually graded essays. Qualitative analysis of the chatbot responses was then performed. Results: Four di erent prompts were given to the arti cial intelligence (AI) driven platform of ChatGPT to grade the summative essays. These were the chatbot's initial responses without grading, the chatbot's response to grading against criteria, the chatbot's response to criteria-wise grading, and the chatbot's response to questions for the di erence in grading. Based on the results, four innovative ways of using AI and machine learning (ML) have been proposed for medical educators: Automated grading, content analysis, plagiarism detection, and formative assessment. ChatGPT provided a comprehensive report with feedback on writing skills, as opposed to manual grading of essays. Conclusion: The chatbot's responses were fascinating and thought-provoking. AI and ML technologies can potentially supplement human grading in the assessment of essays. Medical educators need to embrace AI and ML technology to enhance the standards and quality of medical education, particularly when assessing long and short essay-type questions. Further empirical research and evaluation are needed to con rm their e ectiveness.
Page(s):
595-599
Published:
Journal: Journal of College of Physicians and Surgeons--Pakistan : JCPSP, Volume: 34, Issue: 5, Year: 2024
Keywords:
Formative assessment
,
Machine learning
,
Arti cial intelligence
,
ChatGPT
,
Essays