让视觉模型学会看懂视频事件的时间顺序。
Incentivizing Temporal-Awareness in Egocentric Video Understanding Models
- 用强化学习对比有序与乱序视频帧输出，奖励时间连贯推理。
- 在五个真实场景数据集上提升事件定位与因果推理准确率。
- 适合需要理解动作时序的智能助手、自动驾驶等应用。
多模态大语言模型在视觉理解中表现强劲，但在第一人称视频场景中常缺乏时间感知能力，因其训练目标未显式奖励时间推理，反而依赖帧级空间捷径。为此，我们提出时间全局策略优化（TGPO），一种基于可验证奖励的强化学习算法，通过对比时序有序与打乱视频帧的模型输出，生成校准且全局归一化的奖励信号，明确鼓励时间连贯推理。结合GRPO和GSPO，TGPO支持冷启动强化学习训练，并有效抑制现有模型习得的空间捷径行为。在五个第一人称视频基准上的实验表明，TGPO持续提升时间定位与因果一致性，优于以往基于强化学习的视频推理方法。结果表明，TGPO为构建时间鲁棒的多模态大模型提供了一条简单且可扩展的路径。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness, particularly in egocentric settings where reasoning depends on the correct ordering and evolution of events. This deficiency stems in part from training objectives that fail to explicitly reward temporal reasoning and instead rely on frame-level spatial shortcuts. To address this limitation, we propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm designed to incentivize temporal awareness in MLLMs. TGPO contrasts model outputs generated from temporally ordered versus shuffled video frames to derive calibrated, globally normalized reward signals that explicitly favor temporally coherent reasoning. Integrated with GRPO and GSPO, TGPO supports cold-start RL training and effectively suppresses spatial shortcut behaviors learned by existing MLLMs. Experiments across five egocentric video benchmarks demonstrate that TGPO consistently improves temporal grounding and causal coherence, outperforming prior RL-based video reasoning approaches. Our results suggest that TGPO offers a simple and scalable pathway toward temporally robust MLLMs for egocentric video understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。