No Detail Found!

Quantifying Similarities: Oncology Documents from Google Bard and Chat GPT

Author(s)
Muhammad Shumail Naveed Department of Computer Science & Information Technology, University of Baluchistan, Quetta, Pakistan
Abstract
Large language models hold immense promise for the future of text generation. Google Bard and ChatGPT, two prominent large language models originating from different research laboratories, have been subjects of various studies since their introduction. Despite numerous perspectives explored in the studies, none has specifically delved into the analysis of the similarity between texts generated by these models within the same category. This study addresses this gap by comparing the document generation capabilities of Google Bard and ChatGPT. The analysis focuses on topic-wise comparable documents related to oncology. In this study, 50 oncology-related documents generated by Google Bard are juxtaposed with equivalent topic-wise documents produced by ChatGPT, utilizing both cosine similarity and Jaccard similarity for comparison. The analysis employed statistical tests including the Kolmogorov-Smirnov test, Shapiro-Wilk test, and the one-sample Wilcoxon signed-rank test. The findings revealed a significant level of resemblance among the documents generated by both models: cosine similarity (mean = 0.66, std. dev. = 0.11, min = 0.23, max = 0.80) and Jaccard similarity (mean = 0.88, std. dev. = 0.06, min = 0.7, max = 1.0). This suggests a probable commonality in their training datasets or sources of oncology-related information. The study also posited that the observed similarity could be attributed to the probabilistic nature of language models and the potential for overfitting during their training processes. This study stands out for offering a unique direction and outcomes that pave the way for further exploration in the domain of large language models.
Publication Details
Page(s) 773-786
DOI DOI not available
Published Journal: International Journal of Innovations in Science & Technology, Volume: 5, Issue: 4, Year: 2023

Request the full text from PASTIC

Keywords
Cosine Similarity, large language models, ChatGPT, Google Bard, Jaccard Similarity.
0

Citations

0

Downloads

61

Views