TM-BERT: A Twitter Modified BERT for Sentiment Analysis on Covid-19 Vaccination Tweets | IEEE International Conference on Digital Futures and Transformative Technologies (ICoDT2) May 24-26, 2022 (Book of Abstracts) 2022

Author(s)

Muhammad Talha Riaz National University of Sciences and Technology Islamabad, Pakistan

Muhammad Shah Jahan National University of Sciences and Technology Islamabad, Pakistan

Sajid Gul Khawaja National University of Sciences and Technology Islamabad, Pakistan

Arslan Shaukat National University of Sciences and Technology Islamabad, Pakistan

Jahan Zeb National University of Sciences and Technology Islamabad, Pakistan

Abstract

In transfer learning a model is pre-trained on a large unsupervised dataset and then fine-tuned on domain specific downstream tasks. BERT is the first true-natured deep bidirectional language model which reads the input from both sides of input to better understand the context of a sentence by solely relying on the Attention mechanism. This study presents a Twitter Modified BERT (TM-BERT) based upon Transformer architecture. It has also developed a new Covid-19 Vaccination Sentiment Analysis Task (CV-SAT) and a COVID19 unsupervised pre-training dataset containing (70K) tweets. BERT achieved (0.70) and (0.76) accuracy when fine-tuned on CV-SAT, whereas TM-BERT achieved (0.89), a (19%) and (13%) accuracy over BERT. Another enhancement introduced is in terms of time efficiency as BERT takes (64) hours of pretraining while TM-BERT takes only (17) hours and still produces (19%) improvement even after pretrained on four (4) times fewer data.

Publication Details

Page(s) 1-1

DOI DOI not available

Published Journal: IEEE International Conference on Digital Futures and Transformative Technologies (ICoDT2) May 24-26, 2022 (Book of Abstracts), Volume: 1, Issue: 1, Year: 2022

Keywords

Sentiment analysis TMBERT Covid19 Vaccination Tweets

References

References are not available for this document.

Citations

Citations are not available for this document.