arXiv:2608.26193cs.AI2026-08被引 1

让AI看懂人物情绪与行为,还能自动生成可验证的推理过程。

AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenes

论文配图:AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenes
图 1 · 摘自论文原文
  • 用强化学习引导模型关注人脸微表情和肢体语言等关键线索。
  • 在情感识别任务上提升4.66%,时序敏感任务提升14.29%。
  • 适合需要可解释、可审计的情感分析场景,如社交互动研究。

多模态大语言模型在视觉问答和场景理解上表现优异,但情感推理仍易受捷径行为影响:模型可能给出正确答案却忽视人物中心线索(如微表情、身体语言),削弱可追溯性与外部验证能力。现有强化学习方法多奖励上下文或逻辑连贯性,未显式强制关注人类证据;且以LLM为裁判评分常出现分数聚集,降低奖励区分度。我们提出AffectOmni,一种基于GRPO训练的可验证情感推理框架。引入‘人物聚焦’与‘时间顺序’奖励,促使模型选择人物中心证据并进行时序结构化推理,并采用组内对比评分机制生成更稳定、可区分的奖励信号。为实现验证,思维摘要器将自由文本推理转化为可执行的证据指令,通过SAM3映射至像素级证据区域,提供训练外可审计接口。在IntentBench、Daily Omni和WorldSense数据集上的实验显示,相比开源7B规模基线,该模型持续取得提升,情感识别提升4.66%,时序敏感任务提升14.29%。代码已开源。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) achieve strong performance on VQA and scene understanding, yet affective reasoning remains vulnerable to shortcut behavior. Models may predict correct answers while neglecting people-centric cues such as micro expressions and body language, which weakens traceability and external verification. Prior reinforcement learning approaches mainly reward context or logical coherence without explicitly enforcing attention to human evidence. In addition, LLM as a Judge scoring often suffers from score clustering, which reduces reward discriminability. We propose AffectOmni, a GRPO trained framework for verifiable affective reasoning. AffectOmni introduces People Focus and Temporal Order rewards to encourage people-centric evidence selection and temporally structured reasoning, and it adopts within-group comparative scoring to produce more stable and discriminative reward signals. For verification, a Thinking Summarizer converts free form rationales into executable evidence instructions, which are grounded into pixel level evidence regions via SAM3 to provide an externally auditable interface outside the training loop. Experiments on IntentBench, Daily Omni, and WorldSense show consistent improvements over open source 7B scale baselines, including gains of 4.66% on emotion recognition and +14.29% on temporally sensitive tasks. Code is available at https://github.com/eliot127825-rgb/AffectOmni_nobody.

情感推理可验证多模态强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。