No Detail Found!

Investigating Effective Transfer Learning Strategies for Natural Language Processing Tasks in Low-Resource Languages

Author(s)
Ayesha Shahid Social Sciences University of Ankara TRNC and PhD Scholar at Cyprus International University TRNC
Shanza Chaudhari (ELT) at the University of Kyrenia, TRNC and M.Phil Scholar at Final International University, TRNC/Cyprus
Anwar Ali Sanjrani Department of Computer Science and Information Technology University of Balochistan Quetta,, Pakistan
Abstract
This research investigates effective transfer learning strategies for Natural Language Processing (NLP) tasks in low-resource languages, focusing on the Pakistani context. Through a comprehensive methodology, including literature review, data collection, and experimentation with pre-trained models like mBERT and XLM-R, the study demonstrates the potential of transfer learning to enhance model performance for Urdu, Panjabi, Balochi, Pashto, and Sindhi. The findings reveal significant improvements in accuracy and F1 scores through fine-tuning and data augmentation. Qualitative analyses, including error inspection and user feedback, highlight the necessity for ongoing refinements in model usability, particularly concerning cultural nuances and real-time feedback mechanisms.
Publication Details
Page(s) 1868-1880
DOI DOI not available
Published Journal: International Journal of Communication Networks and Information Security, Volume: 16, Issue: 3, Year: 2024

Request the full text from PASTIC

Keywords
Urdu, Sindhi, Transfer Learning, natural language processing, Pashto, Panjabi, mBERT, XLM-R, Balochi, LowResource Languages
References
Ahmad F.,Khan A. .2021 .Transfer Learning in Low-Resource Languages: A Survey. Language Resources and Evaluation, 55(1) : 1-20.
Ali S.,Ahmed M. .2021 .Interpretability in NLP Models: Challenges and Opportunities. Journal of Artificial Intelligence Research, 70 : 1-25.
Baloch M.,Iqbal N. .2022 .User-Centric Design in NLP Applications: Insights from Low-Resource Contexts. Computers in Human Behavior, 118 : 106709.
Bahl P.,Singh S.,Kumar M. .2022 .User Feedback in NLP: Insights and Implications. Journal of Language Technology, 12(3) : 45-60.
Bashir H.,Siddique M. .2021 .Data Augmentation Techniques for NLP: A Review. Journal of Information Science, 47(3) : 302-315.
Bhatia A.,Mehta V.,Verma A. .2021 .Cultural Nuances in Sentiment Analysis: A Case Study of South Asian Languages. International Journal of Information Management, 57 : 102-115.
Conneau A.,Khandelwal U.,Goyal N. .2020 .Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, : 8440-8451.
Conneau A.,Lample G. .2020 .Cross-lingual Language Model Pretraining. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, : 7181-7195.
Chen S.,Wang H. .2022 .Multilingual Pre-trained Models for Low-Resource Languages. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, : 304-310.
Devlin J.,Chang M. W.,Lee K.,Toutanova K. .2019 .BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, : 4171-4186.
Farooq U.,Muneeb A. .2021 .The Importance of User Feedback in NLP Model Development. International Journal of Computer Applications, 177(21) : 18-24.
Gao T.,Yang D.,Chen Y. .2020 .SimCSE: Simple Contrastive Learning of Sentence Embeddings. arXiv preprint arXiv:2104, : 08836.
Gulati A. .2021 .Transformers for Speech Processing: A Review. IEEE Transactions on Audio, Speech, and Language Processing, 29 : 352-367.
Huang M.,Li J.,Zhang Y. .2022 .A Comparative Study of Transfer Learning for NLP Tasks in Low-Resource Languages. Journal of Artificial Intelligence Research, 73 : 313-339.
Hussain A.,Shah M. .2021 .Challenges in NLP for Low-Resource Languages: A Case Study. Journal of Language and Linguistics, 10(1) : 123-138.
Javed M.,Bukhari S. .2022 .Enhancing Sentiment Analysis in Low-Resource Languages Using Transfer Learning. Expert Systems with Applications, 200 : 117032.
Khan R.,Iqbal Z. .2023 .The Role of Interpretability in NLP: Building Trust Among Users. Artificial Intelligence Review, 56(3) : 305-320.
Kocmi T.,Bojar O. .2021 .A Multilingual Perspective on Zero-Shot Learning for NLP Tasks. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, : 25-30.
Kohavi R.,Selection Model .1995 .A Study of Cross-Validation and Bootstrap for Accuracy Estimation. International Joint Conference on Artificial Intelligence, : 1137-1145.
Kumar R.,Jain P.,Sharma N. .2023 .Exploring User Trust in AI-Driven NLP Applications. , 145 : 107-121.
Kumar S.,Verma P. .2021 .Effective Strategies for NLP in Low-Resource Languages: A Review. Journal of Language Technology, 13(2) : 89-102.
Li X.,Zhang H. .2020 .Zero-Shot Learning for Named Entity Recognition. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, : 180-189.
Nadeem M.,Khan F. .2022 .Misclassification in Sentiment Analysis: User Trust and System Reliability. International Journal of Human-Computer Interaction, 38(4) : 284-298.
Pérez J.,Torres M. .2020 .Transfer Learning for Low-Resource Languages: A Systematic Review. ACM Transactions on Asian and Low-Resource Language Information Processing, 19(1) : 1-19.
Raza B.,Fatima S. .2023 .Addressing Dialectal Variations in Low-Resource NLP Tasks. Journal of Language and Linguistic Studies, 19(1) : 1-15.
Ruder S. .2021 .Transfer Learning in Natural Language Processing: A Survey. Journal of Artificial Intelligence Research, 70 : 9-24.
Shah P.,Ahmed R. .2022 .Collaborative Approaches to Enhance NLP for Low-Resource Languages. Journal of Natural Language Engineering, 28(5) : 1033-1050.
Shabir G.,Iftikhar R. .2022 .Enhancing Language Processing for Urdu: A Transfer Learning Approach. International Journal of Speech Technology, 25(2) : 387-401.
Shahid M.,Ali H. .2023 .Back-Translation in NLP: A Case Study on Urdu Sentiment Analysis. Computational Linguistics, 49(1) : 1-18.
Schick T.,Schütze H. .2009 .Exploiting OpenAI's GPT-3 for Few-Shot and Zero-Shot Learning. , : .
Schick T.,Schütze H. .2021 .Exploiting the Sentence Structure for Few-Shot Learning. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics, : 1033-1040.
Sennrich R.,Haddow B.,Birch A. .2016 .. Edinburgh Neural Machine Translation Systems for WMT 16. Proceedings of the First Conference on Machine Translation, : 371-376.
Wei J.,Zou K. .1901 .EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. , : .
Zhao Y.,Li X.,Zhang Q. .2023 .Improving Sentiment Analysis in Low-Resource Contexts: A User-Centric Approach. Journal of Language and Social Psychology, 42(2) : 178-196.
Zhang Y.,Wang X.,He Y. .2021 .Leveraging Cross-Lingual Transfer Learning for LowResource Language Processing. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, : 4420-4430.
Zhou Y.,Xu C. .2023 .Multilingual Models for Sentiment Analysis: Opportunities and Challenges. Journal of Computer Science and Technology, 38(1) : 45-60.
Zhang L.,Yang Y. .2023 .Unsupervised Learning for Low-Resource NLP Tasks: A Survey. Artificial Intelligence Review, 56(2) : 293-311.
0

Citations

0

Downloads

74

Views