混合CNN与Transformer的模型提升语音情感识别准确率
EmoFormer: A Text-Independent Speech Emotion Recognition using a Hybrid Transformer-CNN model
- 结合CNN与Transformer捕捉语音情绪特征
- 5类情绪识别准确率达90%，7类降至83%
- 适用于无特定说话人和语境的语音情感分析
语音情感识别是人机交互中的关键研究方向。尽管已有大量工作，但现有先进模型在语音和说话人无关的数据上仍难以准确识别情绪。为此，本文提出EmoFormer，一种融合卷积神经网络（CNN）与Transformer编码器的混合模型，用于捕捉此类数据中的情绪模式。该模型在META发布的表达式消音录音语料库（EARS）上进行训练与测试，并采用MFCCs和x-vectors两种特征提取方法。在包含5、7、10和23个情绪类别的不同集合上进行了评估。结果显示，5类情绪识别时准确率达到90%，精确率为0.92，召回率与F1分数均为0.91；随着情绪类别增加，性能下降，7类情绪下准确率为83%，优于基线网络的70%。研究证明，结合CNN与Transformer架构在使用MFCC特征时对语音情感识别具有显著效果。
原文摘要 · Abstract (English)
Speech Emotion Recognition is a crucial area of research in human-computer interaction. While significant work has been done in this field, many state-of-the-art networks struggle to accurately recognize emotions in speech when the data is both speech and speaker-independent. To address this limitation, this study proposes, EmoFormer, a hybrid model combining CNNs (CNNs) with Transformer encoders to capture emotion patterns in speech data for such independent datasets. The EmoFormer network was trained and tested using the Expressive Anechoic Recordings of Speech (EARS) dataset, recently released by META. We experimented with two feature extraction techniques: MFCCs and x-vectors. The model was evaluated on different emotion sets comprising 5, 7, 10, and 23 distinct categories. The results demonstrate that the model achieved its best performance with five emotions, attaining an accuracy of 90%, a precision of 0.92, a recall, and an F1-score of 0.91. However, performance decreased as the number of emotions increased, with an accuracy of 83% for seven emotions compared to 70% for the baseline network. This study highlights the effectiveness of combining CNNs and Transformer-based architectures for emotion recognition from speech, particularly when using MFCC features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。