用强化学习自动选好语音情绪样本，提升无监督识别效果
B-GRPO: Unsupervised Speech Emotion Recognition based on Batched-Group Relative Policy Optimization
- 将样本选择建模为序列决策，用批内相对优势优化策略
- 在多个数据集上比基线提升19.8%的准确率
- 适合标注稀缺场景下的语音情绪分析研究
无监督语音情绪识别旨在解决情绪语音数据稀疏和标注偏差问题。强化学习通过规则或模型生成的验证函数提升性能，无需依赖人工标注。本文将学习过程中的样本选择视为长期决策过程，将是否选择某样本作为动作，从而将强化学习应用于语音情绪识别中的样本质量评估。提出改进的批组相对策略优化（B-GRPO），将一批样本视为一组，以该组平均奖励为基准计算优势。不同于原GRPO使用可验证奖励函数，本文引入自奖励与教师奖励函数，激励模型输出高置信度结果。实验表明，所提方法相比无强化学习的基线模型性能提升19.8%。
原文摘要 · Abstract (English)
Unsupervised speech emotion recognition (SER) focuses on addressing the problem of data sparsity and annotation bias of emotional speech. Reinforcement learning (RL) is a promising method which enhances the performance through rule-based or model-based verification functions rather than human annotations. We treat the sample selection during the learning process as a long-term procedure and whether to select a sample as the action to make policy, thus achieving the application of RL to measure sample quality in SER. We propose a modified Group Relative Policy Optimization (GRPO) to adapt it to classification problems, which takes the samples in a batch as a group and uses the average reward of these samples as the baseline to calculate the advantage. And rather than using a verifiable reward function as in GRPO, we put forward self-reward functions and teacher-reward functions to encourage the model to produce high-confidence outputs. Experiments indicate that the proposed method improves the performance of baseline without RL by 19.8%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。