No Detail Found!

Improving Part of Speech Tagging for Learner English.

Author(s)
Taiwo Kolajo Department of Computer Science, Federal University Lokoja, Thomas Adewumi University, Nigeria
Emeka Ogbuju Department of Computer Science, Federal University Lokoja, Thomas Adewumi University, Nigeria
Abubakar Aliyu Department of Computer Science, Federal University Lokoja, Thomas Adewumi University, Nigeria
Victoria Yemi-Peters Department of Computer Science, Federal University Lokoja, Thomas Adewumi University, Nigeria
Kayode Atteh Department of Computer Science, Federal University Lokoja, Thomas Adewumi University, Nigeria
Francisca Oladipo Department of Computer Science, Federal University Lokoja, Thomas Adewumi University, Nigeria
Abstract
As advanced algorithms for natural language processing are being developed, fundamental tasks like part of speech tagging remain crucial and challenging. As a crucial step in natural language processing, part of speech tagging assigns each word in a phrase to the correct part of speech. Spelling mistakes are a frequent blunder we make when utilizing computer software. Spelling mistakes have a negative impact on part-of-speech tagging performance. In this paper, the integration of spell-checking into part-of-speech tagging was proposed. In our comparison of four spell checkers-Norvig, Jamspell, Hunspell, and Dummy-Jamspell outperformed the others in terms of accuracy, speed, and the number of words it broke during the correction. This influenced our decision to use Jamspell to correct spelling mistakes. Treebank, Brown, and Conll2000 NLTK corpora were utilized for the project's Part of speech tagging. We applied a softmax function on the hidden state of the LSTM after passing an LSTM over the tokens to make predictions. The outcome is a vector of tag scores from which we can determine the projected tag for a word based on the highest value in this distribution of tag scores. The results show an accuracy of 97%, a macro average of 98% and a weighted average of 97% when tested on 212 tagged sentences after spelling correction was applied. The results also show that integrating spell-checking into part-of-speech tagging process improves its accuracy.
Publication Details
Page(s) 6-13
DOI DOI not available
Published Journal: University of Sindh Journal of Information and Communication Technology, Volume: 7, Issue: 1, Year: 2023

Request the full text from PASTIC

Keywords
LSTM, Part-of-speech tagging, Learner English, Spelling checker
0

Citations

0

Downloads

24

Views