用结构化事件证据提升视频推理准确率,让模型更像人一样思考。
STEER: Structured Event Evidence for Video Reasoning via Multi-Objective Reinforcement Learning
- 构建时间有序的事件架构,提取关键事件与依赖关系。
- 在4个任务上超越7B模型,仅用一半输入帧达到同等效果。
- 适合需要可解释视频理解的AI研发与多模态系统开发者。
人类理解视频动态依赖于对实体、动作和时间关系的结构化表征,而现有视频大模型直接对原始视觉标记进行无结构思维链推理,导致关键时间线索被冗长叙述淹没,事件层级结构被忽略。本文提出结构化事件证据(Structured Event Evidence),将视频表示为紧凑的时间有序事件图谱,捕捉显著事件的关键属性及事件间时间依赖,通过约束性验证实现基于证据的推理。该设计提升推理简洁性与可解释性,减少无约束思维链中的偏差漂移。为训练此范式,构建了含四阶段渐进流程的STEER-60K数据集:证据训练、格式预热、思维预热和强化学习后训练。强化学习中,思维链长度与任务准确率常冲突,且难样本奖励稀疏,导致策略忽略挑战实例。本文将其建模为多目标帕累托最优问题,提出帕累托前沿引导的优势平衡方法(P-FAB),动态化解奖励冲突,在帕累托前沿上识别均衡优化方向。最终模型STEER-4B在视频理解任务上媲美7B规模基线,仅需一半输入帧。代码与数据将公开。
原文摘要 · Abstract (English)
Human understanding of video dynamics relies on forming structured representations of entities, actions, and temporal relations before engaging in abstract reasoning. In contrast, existing Video-LLMs apply unstructured chain-of-thought directly to raw visual tokens, where critical temporal cues are buried in verbose narration and event-level structure is largely overlooked. We propose Structured Event Evidence, which represents a video as a compact, time-ordered event schema capturing salient events with key attributes and inter-event temporal dependencies, enabling evidence-grounded reasoning through a constrained verification process. This design promotes concise, interpretable reasoning while reducing the drift typical of unconstrained chain-of-thought. To train models under this paradigm, we introduce STEER-60K, a dataset with a four-stage progressive pipeline: evidence training, format warm-start, thinking warm-start, and RL post-training. During RL, CoT length and task accuracy often conflict while rewards for hard samples are too sparse, causing the policy to neglect challenging instances. We formulate this as a multi-objective Pareto optimality problem and propose Pareto-Frontier guided Advantage Balancing (P-FAB), which dynamically resolves reward conflicts and identifies balanced optimization directions along the Pareto frontier. The resulting model STEER-4B rivals 7B-scale baselines on video understanding tasks with half the input frames Code and data will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。