Sentiment Analysis and Emotion Recognition from Speech Using Universal Speech Representations.

Publisher: MDPI Country of Publication: Switzerland NLM ID: 101204366 Publication Model: Electronic Cited Medium: Internet ISSN: 1424-8220 (Electronic) Linking ISSN: 14248220 NLM ISO Abbreviation: Sensors (Basel) Subsets: MEDLINE

Original Publication: Basel, Switzerland : MDPI, c2000-

Sentiment Analysis* ; Speech*; Attitude ; Emotions ; Humans ; Language

The study of understanding sentiment and emotion in speech is a challenging task in human multimodal language. However, in certain cases, such as telephone calls, only audio data can be obtained. In this study, we independently evaluated sentiment analysis and emotion recognition from speech using recent self-supervised learning models-specifically, universal speech representations with speaker-aware pre-training models. Three different sizes of universal models were evaluated for three sentiment tasks and an emotion task. The evaluation revealed that the best results were obtained with two classes of sentiment analysis, based on both weighted and unweighted accuracy scores (81% and 73%). This binary classification with unimodal acoustic analysis also performed competitively compared to previous methods which used multimodal fusion. The models failed to make accurate predictionsin an emotion recognition task and in sentiment analysis tasks with higher numbers of classes. The unbalanced property of the datasets may also have contributed to the performance degradations observed in the six-class emotion, three-class sentiment, and seven-class sentiment tasks.

Perspect Psychol Sci. 2007 Sep;2(3):260-80. (PMID: 26151969)
Proc Conf Empir Methods Nat Lang Process. 2020 Nov;2020:1823-1833. (PMID: 33969363)
Proc Conf AAAI Artif Intell. 2019 Jul;33(1):7216-7223. (PMID: 32219010)
Proc Conf Assoc Comput Linguist Meet. 2019 Jul;2019:6558-6569. (PMID: 32362720)
Front Psychol. 2019 Nov 08;10:2476. (PMID: 31787911)

JPNP20006 New Energy and Industrial Technology Development Organization

Keywords: affective computing; sentiment analysis; sentiment analysis and emotion recognition; speech emotion recognition; universal speech representation

Date Created: 20220909 Date Completed: 20220912 Latest Revision: 20220913

20240513

PMC9460459

10.3390/s22176369

36080828