提出高效微调方法,让语音情感识别更快更省资源。
Efficient Finetuning for Dimensional Speech Emotion Recognition in the Age of Transformers
- 只微调最后三层并用混合精度,速度提升67%。
- 缓存中间表示可提速88%,参数量减少71%。
- 适合资源有限的研究者快速部署情感识别系统。
准确的语音情感识别对人机交互系统至关重要。尽管预训练的Transformer模型(如Wav2Vec 2.0)已显著提升性能,但微调过程仍需大量计算资源。本文聚焦于维度化情感识别(预测激活度与效价),比较了全量微调、部分层微调、混合精度、缓存中间表示及低秩适配(LoRA)等多种策略。结果表明,仅微调最后三层次并结合混合精度,可达到全量微调性能,同时训练速度提升67%;加入中间表示缓存后,训练速度再提升88%,可学习参数减少71%。推荐采用最后三层层微调+混合精度+缓存策略,在保持高精度的同时显著降低资源消耗,使情感识别系统更易被广泛使用。
原文摘要 · Abstract (English)
Accurate speech emotion recognition is essential for developing human-facing systems. Recent advancements have included finetuning large, pretrained transformer models like Wav2Vec 2.0. However, the finetuning process requires substantial computational resources, including high-memory GPUs and significant processing time. As the demand for accurate emotion recognition continues to grow, efficient finetuning approaches are needed to reduce the computational burden. Our study focuses on dimensional emotion recognition, predicting attributes such as activation (calm to excited) and valence (negative to positive). We present various finetuning techniques, including full finetuning, partial finetuning of transformer layers, finetuning with mixed precision, partial finetuning with caching, and low-rank adaptation (LoRA) on the Wav2Vec 2.0 base model. We find that partial finetuning with mixed precision achieves performance comparable to full finetuning while increasing training speed by 67%. Caching intermediate representations further boosts efficiency, yielding an 88% speedup and a 71% reduction in learnable parameters. We recommend finetuning the final three transformer layers in mixed precision to balance performance and training efficiency, and adding intermediate representation caching for optimal speed with minimal performance trade-offs. These findings lower the barriers to finetuning speech emotion recognition systems, making accurate emotion recognition more accessible to a broader range of researchers and practitioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。