轻量级Transformer模型在语音情绪识别中表现优异，适合边缘设备实时应用。
Emotion Detection in Speech Using Lightweight and Transformer-Based Models: A Comparative and Ablation Study
- 对比DistilHuBERT与PaSST等轻量Transformer模型，使用MFCC特征分类六类情绪。
- DistilHuBERT准确率达70.64%，F1为70.36%，模型仅0.02MB，优于基线与其它变体。
- 愤怒情绪最易识别，厌恶最难；轻量化模型适合部署于资源受限的终端设备。
语音情绪识别对发展共情式人机交互系统至关重要。本文基于CREMA-D数据集，对轻量级Transformer模型DistilHuBERT和PaSST进行比较分析，以识别六类核心情绪，并与使用MFCC特征的传统CNN-LSTM基线模型对比。结果表明，DistilHuBERT在准确率（70.64%）和F1分数（70.36%）上均领先，且模型大小仅为0.02MB。此外，我们对PaSST的三种分类头（Linear、MLP、Attentive Pooling）进行了消融实验，发现含MLP头的版本表现最佳，但仍不及DistilHuBERT。各类情绪中，愤怒识别最准确，而厌恶最难检测。研究显示，如DistilHuBERT等轻量级Transformer模型为边缘设备上的实时语音情绪识别提供了有效方案。代码已开源。
原文摘要 · Abstract (English)
Emotion recognition from speech plays a vital role in the development of empathetic human-computer interaction systems. This paper presents a comparative analysis of lightweight transformer-based models, DistilHuBERT and PaSST, by classifying six core emotions from the CREMA-D dataset. We benchmark their performance against a traditional CNN-LSTM baseline model using MFCC features. DistilHuBERT demonstrates superior accuracy (70.64%) and F1 score (70.36%) while maintaining an exceptionally small model size (0.02 MB), outperforming both PaSST and the baseline. Furthermore, we conducted an ablation study on three variants of the PaSST, Linear, MLP, and Attentive Pooling heads, to understand the effect of classification head architecture on model performance. Our results indicate that PaSST with an MLP head yields the best performance among its variants but still falls short of DistilHuBERT. Among the emotion classes, angry is consistently the most accurately detected, while disgust remains the most challenging. These findings suggest that lightweight transformers like DistilHuBERT offer a compelling solution for real-time speech emotion recognition on edge devices. The code is available at: https://github.com/luckymaduabuchi/Emotion-detection-.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。