用Whisper提取波斯语情绪特征,降维后更高效准确
A Study of ASR Adaptation and Representation Dimensionality Reduction in Persian Speech Emotion Recognition Using Whisper
- 用PCA降低Whisper语音嵌入维度,省去可训练层
- 在ShEMO数据集上准确率提升,训练速度更快内存更少
- 适合资源匮乏语言的情绪识别研究者参考
低资源语言的语音情绪识别(SER)因标注数据有限而困难。本文研究使用Whisper模型进行波斯语情绪识别,重点探索表示维度压缩与语言特异性模型适配。提出一种框架:从Whisper编码器提取帧级嵌入,通过PCA降维,避免使用可学习投影层,显著减少可训练参数。降维后的表示采用基于注意力的池化聚合,再由轻量预测头分类。同时,探究在波斯语自动语音识别(ASR)任务上微调Whisper对下游SER性能的影响。在独立说话人评估协议下的ShEMO数据集实验表明,基于PCA的维度压缩持续提升情绪识别性能,同时降低训练延迟与内存占用;而ASR微调仅带来小幅增益,表明在当前条件下语言适应对情绪相关表征的迁移作用有限。这些发现为高效利用大预训练语音模型实现低资源语言情绪识别提供了实用洞见。
原文摘要 · Abstract (English)
Speech Emotion Recognition (SER) in low-resource languages remains a challenging problem due to limited labeled data. In this work, we study the use of Whisper for Persian SER with a particular focus on representation dimensionality reduction and language-specific model adaptation. We propose a SER framework in which frame-level embeddings extracted from the Whisper encoder are reduced in dimensionality using PCA, eliminating the need for learned projection layers and substantially reducing the number of trainable parameters. The reduced representations are aggregated using an attention-based pooling mechanism and classified with a lightweight prediction head. In addition, we investigate whether fine-tuning Whisper on a Persian automatic speech recognition (ASR) task improves downstream SER performance. Experiments conducted on the ShEMO dataset under a speaker-independent evaluation protocol show that PCA-based dimensionality reduction consistently improves emotion recognition performance while reducing training latency and memory usage. ASR fine-tuning yields only modest gains for SER, suggesting limited transfer from language adaptation to emotion-related representations under the evaluated conditions. These findings provide practical insights into the efficient use of large pretrained speech models for emotion recognition in low-resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。