让语音情绪识别可解释，用自然语言说明判断依据。
Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognition
- 将情绪识别转为生成式推理，输出带理由的预测
- 采用多数投票+所有标注者匹配的评分机制，提升公平性
- 人类评估认为生成理由合理且有依据，适合需要透明决策场景
语音情绪识别（SER）通常基于多数投票标签进行训练与评估，虽简化了基准测试，却掩盖了主观性并缺乏预测依据的透明度，忽视有效少数标注且限制可解释性。本文提出一种可解释的语音语言模型（SpeechLM）框架，将SER视为生成式推理任务：给定一段语音，模型先生成转录文本，再输出情绪标签及基于词汇和声学线索的自然语言理由。理由由具备推理能力的教师大模型生成，并作为中间监督信号，与多数标签共同用于微调。不同于以往仅关注分类准确率的工作，本研究在保持竞争性性能的同时提升可解释性。为此，我们引入注释者感知评分，只要预测匹配任意标注者标签即获奖励。在MSP-Podcast v1.12数据集上，模型相比零样本SpeechLM基线仍保持优势，且生成的理由经人工评估被认为合理且依据充分。结果表明，引入理由监督是实现可解释SER且不牺牲预测质量的可行路径。
原文摘要 · Abstract (English)
Speech Emotion Recognition (SER) is typically trained and evaluated on majority-voted labels, which simplifies benchmarking but masks subjectivity and provides little transparency into why predictions are made. This neglects valid minority annotations and limits interpretability. We propose an explainable Speech Language Model (SpeechLM) framework that frames SER as a generative reasoning task. Given an utterance, the model first produces a transcript, then outputs both an emotion label and a concise natural-language rationale grounded in lexical and acoustic cues. Rationales are generated by a reasoning-capable teacher LLM and used as intermediate supervision, combined with majority labels during fine-tuning. Unlike prior work primarily focused on boosting classification accuracy, we aim to enhance explainability while preserving competitive performance. To this end, we complement majority-label metrics with annotator-aware scoring that credits matches with any annotator label. On MSP-Podcast v1.12, our model maintains improvements over zero-shot SpeechLM baselines, and produces rationales that human evaluators find plausible and well grounded. This demonstrates that incorporating rationale supervision offers a practical path toward interpretable SER without sacrificing predictive quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。