用自监督模型和对抗训练，让语音情绪识别更抗说话人差异
SISER: Speaker-Invariant Speech Emotion Recognition with Entropy-Based Adversarial Training
- 用wav2vec 2.0提取语义特征，结合ECAPA-TDNN做对抗训练抑制说话人信息
- 在IEMOCAP数据集上达到60.63%的无差别准确率，优于基线10个百分点
- 适合需要跨说话人泛化的语音情绪识别系统开发者参考
语音情绪识别（SER）面临标注数据稀缺和说话人差异两大挑战，制约系统泛化能力。现有对抗方法虽缓解说话人差异，却未充分利用强大预训练表征。本文提出SISER框架，将wav2vec 2.0作为特征编码器，ECAPA-TDNN作为说话人判别器，构建基于熵的对抗训练机制。wav2vec 2.0提供丰富的自监督表征，降低对大规模标注数据依赖；ECAPA-TDNN通过更强的对抗信号，有效抑制说话人身份信息。在IEMOCAP数据集上的实验表明，SISER实现60.63%的无差别准确率（UA），显著优于基线（51.15%）和未进行说话人抑制的wav2vec 2.0（56.46%）。消融实验验证了说话人判别器架构选择的关键作用。
原文摘要 · Abstract (English)
Speech emotion recognition (SER) faces two fundamental challenges: scarcity of labeled data and inter-speaker variability, both of which hinder generalization of emotion recognition systems. While prior adversarial approaches address speaker variability, they fall short in leveraging powerful pre-trained representations. We propose SISER (Speaker-Invariant Speech Emotion Recognition), integrating wav2vec 2.0 as a feature encoder and ECAPA-TDNN as a speaker discriminator within an entropy-based adversarial training scheme. wav2vec 2.0 provides rich self-supervised representations that alleviate dependency on large labeled datasets, while ECAPA-TDNN enables suppression of speaker identity via a stronger adversarial signal than shallow classifiers. Evaluated on IEMOCAP, SISER achieves a UA of 60.63%, outperforming the baseline (51.15%) and wav2vec 2.0 without speaker suppression (56.46%), with ablation emphasizing that the choice of speaker classifier architecture is a key factor.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。