让视频大模型学会关注动态变化，而非依赖静态线索。
Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning
- 通过反事实视频对比，训练模型判断答案是否该随画面变化而改变。
- 在3014个反事实视频上测试，性能提升7.7到8.2点，避免误用静态捷径。
- 适合想提升模型理解视频时序与空间动态能力的研究者。
视频大语言模型虽在基准测试中表现优异，却常依赖单帧线索或语言先验，而非跟踪时空动态。此问题在强化学习后训练中更严重，因仅奖励正确性会强化捷径策略。为此，本文提出反事实关系策略优化（CRPO），通过水平翻转和时间倒置生成反事实视频，构建双分支强化学习框架，并引入反事实关系奖励（CRR），要求动态问题的答案在反事实视频中改变，静态问题则保持不变。该机制使捷径策略难以同时在两分支获得高奖励。为评估该能力，我们构建了包含3014个视频的DyBench基准，涵盖可逆动态、运动方向与事件序列，并采用严格配对准确率指标防止固定答案作弊。实验表明，CRPO在时空敏感任务上优于现有方法，同时保持良好通用性能：在Qwen3-VL-8B上，DyBench P-Acc提升+7.7，TimeBlind I-Acc提升+8.2，证明其提升了时空敏感性而非依赖静态线索。
原文摘要 · Abstract (English)
Video large language models (Video LLMs) achieve strong benchmark accuracy, yet often answer video questions through shortcuts such as single-frame cues and language priors rather than by tracking spatiotemporal dynamics. This issue is exacerbated in RL post-training, where correctness-only rewards can further reinforce shortcut policies that obtain high reward without tracking video dynamics. We address this by asking a controlled counterfactual question: if the visual world changed while the question remained fixed, should the answer change or stay the same? Based on this view, we propose \textbf{Counterfactual Relational Policy Optimization (CRPO)}, a dual-branch RL framework for improving \emph{spatiotemporal sensitivity}. CRPO constructs counterfactual videos through horizontal flips and temporal reversals, trains on both original and counterfactual branches, and introduces a \textbf{Counterfactual Relation Reward (CRR)} between their answers. CRR encourages answers to change for dynamic questions and remain unchanged for static questions. This cross-branch constraint makes it difficult for shortcut policies to be consistently rewarded across both branches. To evaluate this property, we introduce \textbf{DyBench}, a paired counterfactual video benchmark with 3,014 videos covering reversible dynamics, moving direction, and event sequence, together with a strict pair-accuracy metric that prevents fixed-answer shortcuts from inflating scores. Experiments show that CRPO outperforms prior RL methods on spatiotemporal-sensitive evaluations while maintaining competitive general video performance. On Qwen3-VL-8B, CRPO improves DyBench P-Acc by +7.7 and TimeBlind I-Acc by +8.2 over the base model, indicating improved spatiotemporal sensitivity rather than stronger reliance on static shortcuts. The project website can be found at https://ddz16.github.io/crpo.github.io/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。