Author(s):
1. Muhammad Talha Riaz:
National University of Sciences and Technology Islamabad, Pakistan
2. Muhammad Shah Jahan:
National University of Sciences and Technology Islamabad, Pakistan
3. Sajid Gul Khawaja:
National University of Sciences and Technology Islamabad, Pakistan
4. Arslan Shaukat:
National University of Sciences and Technology Islamabad, Pakistan
5. Jahan Zeb:
National University of Sciences and Technology Islamabad, Pakistan
Abstract:
In transfer learning a model is pre-trained on a large unsupervised dataset and then fine-tuned on domain specific downstream tasks. BERT is the first true-natured deep bidirectional language model which reads the input from both sides of input to better understand the context of a sentence by solely relying on the Attention mechanism. This study presents a Twitter Modified BERT (TM-BERT) based upon Transformer architecture. It has also developed a new Covid-19 Vaccination Sentiment Analysis Task (CV-SAT) and a COVID19 unsupervised pre-training dataset containing (70K) tweets. BERT achieved (0.70) and (0.76) accuracy when fine-tuned on CV-SAT, whereas TM-BERT achieved (0.89), a (19%) and (13%) accuracy over BERT. Another enhancement introduced is in terms of time efficiency as BERT takes (64) hours of pretraining while TM-BERT takes only (17) hours and still produces (19%) improvement even after pretrained on four (4) times fewer data.
Page(s):
1-1
DOI:
DOI not available
Published:
Journal: IEEE International Conference on Digital Futures and Transformative Technologies (ICoDT2) May 24-26, 2022 (Book of Abstracts), Volume: 1, Issue: 1, Year: 2022
Keywords:
Sentiment analysis
,
TMBERT
,
Covid19 Vaccination Tweets
References:
References are not available for this document.
Citations
Citations are not available for this document.