Speech Emotion Recognition Using Hierarchical Approach Based on Bhattacharyya Distance

Abrar Adil; Sana Ul Haq; Muhammad Saeed Shah; Imtiaz Rasool; Muhammad Kamran

Author(s)

Abrar Adil Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Sana Ul Haq Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Muhammad Saeed Shah Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Imtiaz Rasool Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Muhammad Kamran Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Abstract

In human-human interaction emotions play an essential role in conveying the information apart from the verbal communication. It is quite challenging for machines to recognize human emotions and respond accordingly. Most research in emotion recognition has focused on using the flat approach in which emotions are classified in a single step using a single best set of features. This paper presents a hierarchical approach based on Bhattacharyya distance for human emotion recognition from speech. The basic aim is to improve the emotion classification performance. The analysis is performed using the Surrey AudioVisual Expressed Emotion (SAVEE) database, while various attribute selectors and classification techniques are implemented to obtain the best results. The experimental results showed better performance for the proposed hierarchical approach as compared to flat approach and other state-of-the-art techniques. The best classification accuracy of 78.12% is achieved for the flat approach, while the best performance of 91.66% is obtained for the hierarchical approach using seven emotions of the SAVEE database.

Publication Details

Page(s) 1-14

DOI DOI not available

Published Journal: The Sciencetech, Volume: 5, Issue: 1, Year: 2024

Request the full text from PASTIC

Keywords

Hierarchical Classification, Gain Ratio, Feature selection, Speech emotion recognition, Info Gain, Support Vector Machine

References

Alluhaidan A. S.,Saidani O.,Jahangir R.,Nauman M. A.,Neffati O. S. .2023 .Speech Emotion Recognition through Hybrid Features and Convolutional Neural Network. Applied Sciences, 13(8) : 4750.

Araño K. A.,Gloor P.,Orsenigo C.,Vercellis C. .2021 .When old meets new: emotion recognition from speech signals. Cognitive Computation, 13 : 771-783.

Azhagusundari B.,Thanamani A. S. .2013 .Feature selection based on information gain. International Journal of Innovative Technology and Exploring Engineering, 2(2) : 18-21.

Badr Y.,Mukherjee P.,Thumati S. .2021 .Speech Emotion Recognition using MFCC and Hybrid Neural Networks. International Joint Conference on Computational Intelligence, : 366-373.

Bhanusree Y.,Kumar S. S.,Rao A. K. .2023 .Time-Distributed Attention-Layered Convolution Neural Network with Ensemble Learning using Random Forest Classifier for Speech Emotion Recognition. Journal of Information and Communication Technology, 22(1) : 49-76.

Bhattacharyya A. .1943 .On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of the Calcutta Mathematical Society, 35(2) : 99-109.

Büchlmann P.,Yu B. .2002 .. The Annals of Statistics, 30(4) : 927-961.

Chalapathi M. V.,Kumar M. R.,Sharma N.,Shitharth S. .2022 .Ensemble Learning by High-Dimensional Acoustic Features for Emotion Recognition from Speech Audio Signal. Security and Communication Networks, : 1-10.

Er M. B. .2020 .A Novel Approach for Classification of Speech Emotions Based on Deep and Acoustic Features. IEEE Access, 8 : 221640-221653.

Eyben F.,Wollmer M.,Schuller B. .2009 .OpenEAR-Introducing the Munich Open-Source Emotion and Affect Recognition Toolkit. International Conference on Affective Computing and Intelligent Interaction and Workshops, : 1-6.

Gama J. .2004 .Functional trees. Machine learning, 55 : 219-250.

Ghazi D.,Inkpen D.,Szpakowicz S. .2010 .Hierarchical approach to emotion recognition and classification in texts. Advances in Artificial Intelligence: 23rd Canadian Conference on Artificial Intelligence, : 40-50.

Haag A.,Goronzy S.,Schaich P.,Williams J. .2004 .Emotion recognition using bio-sensors: First steps towards an automatic system. Tutorial and research workshop on affective dialogue systems, : 36-48.

Hall M. A. .1999 .Correlation-based feature selection for machine learning. , : .

Haq S.,,Jackson P. J. .2011 .Multimodal Emotion Recognition. IGI Global., : 398-423.

Jain A.,Nandakumar K.,Ross A. .2005 .Score normalization in multimodal biometric systems. Pattern Recognition, 38(12) : 2270-2285.

Jain Y. K.,Bhandare S. K. .2011 .Min max normalization based data perturbation method for privacy protection. International Journal of Computer & Communication Technology, 2(8) : 45-50.

Khare S. K.,Blanes-Vidal V.,Nadimi E. S.,Acharya U. R. .2024 .Emotion recognition and artificial intelligence: A systematic review (2014-2023) and research recommendations. Information Fusion, 102 : .

Liu S.,McGree J.,Ge Z.,Xie Y. .2016 .Computational and Statistical Methods for Analysing Big Data with Applications. , : .

Liu Z.-T.,Xie Q.,Wu M.,Cao M.,Mei Y.,Mao J.-W. .2018 .Speech emotion recognition based on an improved brain emotion learning model. Neurocomputing, 309 : 145-156.

Mannepalli K.,Sastry P. N.,Suman M. .2022 .Emotion recognition in speech signals using optimization based multi-SVNN classifier. Journal of King, 34(2) : 384-397.

Mao Q.,Dong M.,Huang Z.,Zhan Y. .2014 .Learning salient features for speech emotion recognition using convolutional neural networks. IEEE Transactions on Multimedia, 16(8) : 2203-2213.

Mohan M.,Dhanalakshmi P.,Kumar R. S. .2023 .Speech Emotion Classification Using Ensemble Models with MFCC. Procedia Computer Science, 218 : 1857-1868.

Novais R. M.,Cardoso P. J.,Rodrigues J. M. .2022 .Emotion Classification from Speech by an Ensemble Strategy. International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting InfoExclusion, : 85-90.

Platt J. C. .1999 .Fast training of support vector machines using sequential minimal optimization. Advances in kernel methods, : 185-208.

Tarasov A.,Delany S. J. .2011 .Benchmarking classification models for emotion recognition in natural speech: A multi-corporal study. IEEE International Conference on Automatic Face & Gesture Recognition, : 841-846.

Witten I. H.,Frank E.,Hall M. A.,Pal C. J. .2016 .Data Mining: Practical Machine Learning Tools and Techniques. , : .

Xiao Z.,Dellandrea E.,Dou W.,Chen L. .2007 .Hierarchical classification of emotional speech. IEEE Transactions on Multimedia, 37 : .

Yüncü E.,Hacihabiboglu H.,Bozsahin C. .2014 .Automatic speech emotion recognition using auditory models with binary decision tree and svm. International conference on pattern recognition, : 773-778.

Zhang S.,Li L.,Zhao Z. .2012 .Audio-visual emotion recognition based on facial expression and affective speech. International Conference on Multimedia and Signal Processing, : 46-52.