Improving Hate Speech Detection on Twitter via Advanced TF-IDF Feature Selection and Machine Learning

Jakeer Hussain Vendichutla; Sri Priya Nagula Malyala

Author(s)

Jakeer Hussain Vendichutla Department of Computer Science and Engineering Sree Dattha Group of Institutions, Sheriguda, Telangana

Sri Priya Nagula Malyala Department of Computer Science and Engineering Sree Dattha Group of Institutions, Sheriguda, Telangana

Abstract

Hate speech is defined as any form of communication-whether written, spoken, or symbolic-that expresses discrimination, hostility, or incites violence against individuals or groups based on attributes such as race, religion, ethnicity, gender, sexual orientation, or disability. With the rise of social media platforms like Twitter, which host millions of users and facilitate rapid dissemination of information, hate speech has proliferated. The open nature and massive scale of these platforms make them vulnerable to the spread of harmful and offensive content. Twitter, in particular, generates an enormous volume of tweets daily, making it infeasible for human moderators to manually review and classify each tweet for hate speech. As a result, developing automated systems to detect and mitigate hate speech is crucial. Traditional methods for detecting hate speech, such as lexiconbased approaches, involve using predefined lists of offensive words or discriminatory terms to identify potentially harmful content. While these methods provide a straightforward solution, they are often limited by several factors. Lexicon-based techniques struggle to adapt to the rapidly changing and evolving language used in hate speech, including slang, coded language, and context-dependent expressions. Moreover, they lack the sophistication to accurately differentiate between genuine hate speech and other forms of expression, such as satire, criticism, or discussions about sensitive topics, leading to either underreporting or over-flagging of content. Given the limitations of traditional approaches, more advanced techniques using machine learning have emerged as promising solutions for automating hate speech detection. Machine learning classifiers can learn from large datasets and capture nuanced patterns in language that lexicon-based methods often miss. By training these models on vast amounts of labeled data, they can identify subtle indicators of hate speech and adapt to the evolving nature of online communication. One particularly effective method involves the use of the Term Frequency-Inverse Document Frequency (TF-IDF) approach, which calculates the importance of words within a document relative to a larger corpus. A modified TF-IDF approach, tailored specifically for hate speech detection, can help capture the unique linguistic characteristics associated with hateful content, including offensive terms, tone, and context. This allows the model to focus not only on the presence of certain words but also on their significance in context, improving accuracy in distinguishing hate speech from non-hateful content. Additionally, by integrating techniques such as natural language processing (NLP) and deep learning, the model can further refine its ability to understand context, sentiment, and semantics, making it more effective at recognizing hate speech even when veiled in ambiguous language. These advanced machine learning models, trained on annotated datasets of tweets, provide a more dynamic, scalable, and accurate solution for detecting hate speech on platforms like Twitter.

Publication Details

Page(s) 396-407

DOI DOI not available

Published Journal: International Journal of Communication Networks and Information Security, Volume: 16, Issue: 1, Year: 2024

Request the full text from PASTIC

Keywords

machine learning, twitter, Hate Speech Detection, Advanced TF-IDF Feature Selection

References

Akuma S.,Lubem T.,Adom I.T.,Comparing I.T. . .Bag of Words and TF-IDF with different models. REFERENCES [1] ., : .

.2022 .for hate speech detection from live tweets. Int. j. inf. tecnol. 14, : 3629-3635.

Adi Kusworo,Kusumaningrum Retno . ."Classification of Hate Speech. , (4) : .

Intelligent Systems & Network,Security Systems & Network,Raza Systems & Network .2022 ."Hate speech detection on Twitter using transfer learning." Computer Speech. , : 146-156.

Roy T.,Novel Hate Speech Detection Using Word Cloud S.S. .2022 .. & Language, 74 : 101365.

.2022 .. Ensemble Learning Coupled with Count Vectorizer. Appl. Sci, 12 : 6611.

Hristea F.,Towards F. . .a Benchmarking System for Comparing Automatic Hate Speech. , : .

.2022 .Detection with an Intelligent Baseline Proposal. Mathematics, 10 : 945.

Michael Melese Woldeyohannis. . .. , : .

Conference and Evaluation,Ahmad S.,F. S. .2022 .Improved Hate Speech Detection System Using. , : .

Ebenezer Olumide . ."Automatic hate speech detection using deep neural networks. , (27) : 978.

Doan .,Long-An . .2022 ."An Implementation of Large-Scale Hate Speech Detection System for. and word embedding." Computación y Sistemas 26.2, : 1007-1013.

Social Media Data Streaming . .". , : .

Networks ,Satellite (COMNETSAT). IEEE ,Şahinuç Furkan .2022 ."Large-scale hate speech detection. , : .

Emmanuel Femi .2022 ."Hate speech detection in Twitter using hybrid embeddings and. " arXiv preprint arXiv:2203.01111, : .

Jenq-Haur Wang .2020 ."Vulnerable community identification using hate. and Cybernetics 13, : 485-525.

.2020 .. speech detection on social media." Information Processing & Management 57.3, : 102087.