No Detail Found!

Multilabel-Thai Text Classification with Transformer-Rnn in Thai Banking Classification

Author(s)
Suwika Plubin Department of Statistics, Faculty of Science, Chiang Mai University,Chiang Mai,Thailand
Walaithip Bunyatisai Department of Statistics, Faculty of Science, Chiang Mai University,Chiang Mai,Thailand
Manad Khamkong Department of Statistics, Faculty of Science, Chiang Mai University,Chiang Mai,Thailand
Thanasak Mouktonglang Data Science Research Center, Department of Statistics, Faculty of Science, Chiang Mai University, Chiang Mai, Thailand
Bandhita Plubin Department of Statistics, Faculty of Science, Chiang Mai University,Chiang Mai,Thailand
Abstract
This research presents a novel approach to multilabel Thai text classification within the banking sector using a hybrid Transformer-RNN model. The study focuses on enhancing the accuracy and efficiency of document categorization in Thai language, addressing the complexities inherent to diverse banking documents. The model integrates BERT for pre-training to leverage contextual embeddings and incorporates an RNN to capture sequential dependencies in the text data. Evaluation of the model's performance was conducted using precision, recall, and F1 score metrics over a 10-fold cross-validation setup. The hybrid TransformerRNN model demonstrated robust performance across multiple evaluation metrics. Specifically, it achieved an average precision of 0.823, recall of 0.817, and F1 score of 0.815. These results indicate the model's efficacy in accurately predicting multiple labels associated with various types of Thai banking documents. Comparative analysis against LSTM, CNN, BERT, and Transformer (Encoder-Only) models further validates the superiority of the proposed approach in handling complex multilabel classification tasks in the Thai banking domain. This research underscores the potential of hybrid Transformer-RNN models in advancing multilabel text classification capabilities, particularly in specialized domains like Thai banking. The findings highlight significant improvements in classification accuracy and model robustness, contributing to enhanced document management, regulatory compliance, and customer service within the banking industry. Future research directions could explore ensemble learning techniques, domain adaptation strategies, and the integration of domain-specific knowledge bases to further enhance the model's performance and applicability in real-world scenarios.
Publication Details
Page(s) 1-10
Published Journal: Pakistan Journal of Life and Social Sciences, Volume: 23, Issue: 1, Year: 2025

Request the full text from PASTIC

Keywords
Multilabel Classification Transformer-RNN Thai Text Classification BERT
References
Aroonmanakun W. .2002 .Collocation and Thai word segmentation. In Proceedings of the SNLPOriental COCOSDA, : 70.
Kongsumran N. .2021 .Thai tokenizer invariant classification based on bi-lstm and distilbert encoders (Master's thesis). chula.ac.th/chulaetd/4655, 4655 : .
Cho ,Caglar Gulcehre Yoshua Bengio. .2014 .Learning phrase representations using RNN encoderdecoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), : 1724-1734.
Devlin ,Ming-Wei Chang,Lee Kenton,Toutanova and Kristina .2018 .BERT: Bidirectional Encoder Representations from Transformers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1 : 4171-4186.
Devlin Jacob,Ming-Wei Chang,Lee Kenton,Toutanova and Kristina .2019 .BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1 : 4171-4186.
Hochreiter Jürgen,Schmidhuber Jürgen .1997 .Long short-term memory. Neural Computation, 9(8) : 1735-1780.
Wang J.,Yang Z.,Z. Z. .2024 .Deep pre-training transformers for scientific paper representation. Electronics, 13(2123) : .
Kim .2014 .Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1746 : 1751.
Liu Xipeng,Qiu Xuanjing,Huang Xuanjing .2016 .Recurrent neural network for text classification with multi-task learning. , : .
Sebastiani .2002 .Machine learning in automated text categorization. ACM Computing Surveys, 34(1) : 1-47.
Yin W.,Kann K.,Yu M.,Schütze H. .2017 .Comparative study of CNN and RNN for natural language processing. , : .
Tsoumakas Ioannis Katakis. .2007 .Multi-label classification: An overview. International Journal of Data Warehousing and Mining, 3(3) : 1-13.
Vaswani Aidan N.,Gomez Illia,Polosukhin Illia .2017 .Attention is all you need. Advances in Neural Information Processing Systems, 30 : 5998-6008.
Maia M.,Sales J. E.,Freitas A.,Handschuh S.,Endres M. .2021 .A comparative study of deep neural network models on multi-label text classification in finance. In Proceedings of the 2021 IEEE 15th International Conference on Semantic Computing (ICSC), : 183-190.
Kaewvichit P.,Pongsathornwiwat A. .2024 .Developing multi-label classification model for improving text categorizing problems: A case of Traffy Fondue platform. Science & Technology Asia, 29(2) : 138-147.
0

Citations

0

Downloads

38

Views