No Detail Found!

Performance Evaluation of Large Language Models in Detecting Buccal Mucosal Lesions Using Smartphone-Based Imaging

Author(s)
Madan Kumar Department of Public Health Dentistry, Ragas Dental College and Hospital, Chennai, Tamil Nadu – 600 119, India
S. Rajeshwari ICMR Project, Ragas Dental College and Hospital, Chennai, Tamil Nadu – 600 119, India
S. Savitha Department of Public Health Dentistry, Ragas Dental College and Hospital, Chennai, Tamil Nadu – 600 119, India
C. Lavanya Department of Oral and Maxillofacial pathology, Ragas Dental College and Hospital, Chennai, Tamil Nadu – 600 119, India
K. Ranganathan Department of Oral and Maxillofacial pathology, Ragas Dental College and Hospital, Chennai, Tamil Nadu – 600 119, India
Arthi Balasubramaniam Department of Public Health Dentistry, Ragas Dental College and Hospital, Chennai, Tamil Nadu – 600 119, India
Abstract
Background: Early detection of oral potentially malignant disorders (OPMDs) is crucial for improving outcomes in oral cancer, particularly in resource-limited settings. Recent advances in large language models (LLMs) and smartphone imaging provide new opportunities for AI-driven diagnostic support; however, their use in detecting oral mucosal lesions remains underexplored. Objective: To evaluate and compare the diagnostic performance of Few-shot prompting, RetrievalAugmented Generation (RAG), and RAG with Chain-of-Thought (RAG + COT) models inthe binary classification of smartphone-captured intraoral buccal mucosa images as either normal or abnormal. Methods: Using a standardized smartphone protocol, 250 intraoral images from 125 patients were categorized as normal, variations, or lesions and split equally into training and testing sets. Few-shot prompting used a subset only 10 test images - 5 normal and 5 lesion - which may produce unstable estimates, while RAG and RAG + COT models trained on the full training set. Expert annotations guided COT descriptors. Variation images were used only in model training for RAG and RAG+COT to improve contextual representation but excluded from binary performance evaluation. Performance was evaluated via sensitivity, specificity, accuracy, F1 score, precision, recall, and 95% confidence intervals. Results: Few-shot prompting achieved 80% sensitivity, 100% specificity, 90% accuracy, and an F1 score of 0.88 with wide CIs due to the very small test set. The RAG model, with 54% sensitivity and 91% specificity, showed limited true positive detection. Adding chain-of-thought (RAG + COT) improved sensitivity to 90% and accuracy to 82% (F1: 0.86), though specificity dropped to 64% leading to a higher false-positive rate with potential implications for screening follow-up, however, highlighting the value of structured logical reasoning in enhancing lesion detection. Conclusion: The RAG + COT model outperformed Few-shot and RAG models in mucosal lesion detection, demonstrating high sensitivity and improved diagnostic accuracy. However, its low specificity highlights the need for human review before acting on AI results. Findings are promising but preliminary, requiring validation in larger and more balanced datasets before clinical adoption. Its structured logical reasoning highlights the potential of LLMs with COT prompting to strengthen communitybased oral cancer screening.
Publication Details
Page(s) 102-107
Published Journal: Journal of Pioneering Medical Sciences, Volume: 14, Issue: S2, Year: 2025

Request the full text from PASTIC

Keywords
Artificial intelligence, Oral Cancer, large language models, and Sensitivity and Specificity, Intraoral Imaging, Cancer Early Diagnosis
References
Shrestha A.D.,March A.D. .2020 ."Prevalence and incidence of oral cancer in low- and middle-income countries: A scoping review.". European Journal of Cancer Care, 29(2) : .
Mantena S. .2021 ."Improving community health-care screenings with smartphone-based AI technologies." The Lancet Digital Health. , 3(5) : e280-e282.
Lin C.,Kuo C.F. .2025 ."Roles and potential of large language models in healthcare: A comprehensive review.". Biomedical Journal, 29 April, : .
Schmidl B.,January B. .2025 ."Artificial intelligence for image recognition in diagnosing oral and oropharyngeal cancer and leukoplakia. " Scientific Reports, 15(1) : .
Umer F.,Naved I. Batool and N.,December I. Batool and N. .2024 ."Innovation and application of large language models (LLMs) in dentistry - a scoping review. " BDJ Open, 10(1) : .
Kumar Madan,P.D. Madan .2025 .A Training Manual for Taking Intraoral Photographs for Community Health Workers [manual]. : LD-20250166075., : .
Gao Y.,December Y. .2023 ."Retrieval-augmented generation for large language models: A survey.". arXiv preprint arXiv:2312.10997, 2(1) : .
Huang ,Zhijian ,October 04 .2024 .Making large language models better planners with reasoning-decision alignment. ” Lecture Notes in Computer Science, September, 09(1) : 5-90.
0

Citations

0

Downloads

65

Views