用前沿模型生成的推理路径做训练辅助,让AI更懂人类情感决策。
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing

- 用前沿模型的推理路径作训练时特权信息,不直接模仿其输出。
- 在MER-UniBench上达84.19分,性能超越现有方法。
- 适合需要可解释性与安全性的对话系统研发者。
多模态大语言模型在复杂推理任务中常受奖励稀疏性困扰,尤其在涉及情绪、意图等主观因素的人机交互场景中,高质量思维链(CoT)标注成本高昂。尽管已有数据集提供专家标注,但直接用于监督微调易导致捷径学习,且缺乏透明度。为此,我们提出OmniOPSD:一种基于推理路径特权的在线自蒸馏框架。该方法将前沿模型生成的、具备感知意识的推理路径作为训练时的教师侧特权信息,而非学生模仿目标。学生从原始输入自主采样轨迹,教师则对相同标记进行评分并提供细粒度的令牌级监督。学生学习自身轨迹分布,无需直接模仿前沿模型输出;推理阶段无需标签、推理路径或闭源模型访问。在MER-UniBench上的实验表明,OmniOPSD达到84.19的平均得分,显著优于现有方法,消融实验进一步验证了推理路径引导的有效性。
原文摘要 · Abstract (English)
Reinforcement learning for multimodal large language models (MLLMs) is often hindered by severe reward sparsity in complex reasoning tasks. This challenge is particularly pronounced in human-centered scenarios involving states, emotions, intentions, and behaviors, where heterogeneous multimodal signals and subjective human factors make high-quality chain-of-thought (CoT) annotations expensive and difficult to obtain. Although many multimodal datasets provide expert-annotated ground-truth labels, directly using these labels for supervised fine-tuning may encourage shortcut learning in multimodal perception and provides limited transparency for safety-critical human--AI interaction. To address these limitations, we propose OmniOPSD, a Rationale-Privileged On-Policy Self-Distillation framework that uses frontier-generated rationales as teacher-side privileged evidence rather than student imitation targets. OmniOPSD uses frontier-generated evidence-aware rationales only as training-time privileged evidence context for a local teacher. The student samples its own rollout from the original multimodal input, while the rationale-privileged teacher scores the same tokens and provides dense token-level supervision. Thus, the student learns on its own trajectory distribution without directly imitating frontier-model completions, and inference requires no labels, rationales, CoT annotations, or closed-source model access. Experiments on MER-UniBench show that OmniOPSD achieves state-of-the-art performance with an average score of $84.19$, and ablations further support the value of rationale-privileged teacher guidance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。