用自监督模型自动提取语音情绪特征,提升识别准确率。
Speaker Emotion Recognition: Leveraging Self-Supervised Models for Feature Extraction Using Wav2Vec2 and HuBERT
- 使用Wav2Vec2和HuBERT从原始音频自动提取特征
- 在RAVDESS等5个数据集上表现优异,真实通话场景也有效
- 适合语音情绪分析、人机交互与客服系统应用
语音是人类表达情感最自然的方式,但情绪定义模糊,使说话人情绪识别(SER)成为一项挑战。该任务受说话人差异、背景噪声、情绪复杂性和语调风格影响,广泛应用于教育、医疗、客户服务和人机交互。传统方法如SVM、HMM、KNN已逐步被深度学习取代,常用输入为频谱图或人工特征。本文研究基于自监督的Transformer模型Wav2Vec2和HuBERT,直接从原始音频中自动提取特征,并用于情绪分类。在RAVDESS、SHEMO、SAVEE、AESDD和Emo-DB等权威数据集上验证了有效性,且在真实呼叫中心对话中也表现出高准确性,证明其在实际场景中的可行性。
原文摘要 · Abstract (English)
Speech is the most natural way of expressing ourselves as humans. Identifying emotion from speech is a nontrivial task due to the ambiguous definition of emotion itself. Speaker Emotion Recognition (SER) is essential for understanding human emotional behavior. The SER task is challenging due to the variety of speakers, background noise, complexity of emotions, and speaking styles. It has many applications in education, healthcare, customer service, and Human-Computer Interaction (HCI). Previously, conventional machine learning methods such as SVM, HMM, and KNN have been used for the SER task. In recent years, deep learning methods have become popular, with convolutional neural networks and recurrent neural networks being used for SER tasks. The input of these methods is mostly spectrograms and hand-crafted features. In this work, we study the use of self-supervised transformer-based models, Wav2Vec2 and HuBERT, to determine the emotion of speakers from their voice. The models automatically extract features from raw audio signals, which are then used for the classification task. The proposed solution is evaluated on reputable datasets, including RAVDESS, SHEMO, SAVEE, AESDD, and Emo-DB. The results show the effectiveness of the proposed method on different datasets. Moreover, the model has been used for real-world applications like call center conversations, and the results demonstrate that the model accurately predicts emotions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。