arXiv:2603.17693cs.CV2026-03被引 3

用合成视频训练模型掌握运动方向、速度等时间基础能力,提升视频理解效果。

Learning Transferable Temporal Primitives for Video Reasoning via Synthetic Videos

  • 通过代码生成合成视频,学习运动方向、速度等时间基本单元
  • 7.7K合成样本超越165K真实样本的推理性能
  • 适合需要高效提升视频时序理解能力的研究者

从图像到视频理解的关键挑战在于视觉语言模型需从识别静态模式转向推理动态变化,如运动轨迹、速度变化和状态转移。现有后训练方法受限于两大问题:(1)数据集缺乏时间中心性,答案可仅从关键帧推断,无需全局时序整合;(2)由专有模型生成的数据存在系统性时序感知错误,如混淆运动方向或误判速度。本文提出SynRL框架,通过程序化生成的合成视频教授模型时间基础单元——方向、速度与状态追踪。我们将时间理解分解为短期感知单元(速度、方向)与长期认知单元,基于代码生成7.7K CoT和7K RL样本,并提供逐帧真值标注。尽管训练仅使用简单几何图形,SynRL在15个基准上实现显著提升,涵盖时序定位、复杂推理与通用视频理解。令人惊讶的是,7.7K合成CoT样本性能优于165K真实样本的Video-R1。我们归因于帧间变化追踪、速度对比等基础时序技能的有效迁移。这确立了新范式:通过精心设计的合成数据进行视频时序学习,是更低成本的扩展路径。

原文摘要 · Abstract (English)

The transition from image to video understanding requires vision-language models (VLMs) to shift from recognizing static patterns to reasoning over temporal dynamics such as motion trajectories, speed changes, and state transitions. Yet current post-training methods fall short due to two critical limitations: (1) existing datasets often lack temporal-centricity, where answers can be inferred from isolated keyframes rather than requiring holistic temporal integration; and (2) training data generated by proprietary models contains systematic errors in fundamental temporal perception, such as confusing motion directions or misjudging speeds. We introduce SynRL, a post-training framework that teaches models temporal primitives, the fundamental building blocks of temporal understanding including direction, speed, and state tracking. Our key insight is that these abstract primitives, learned from programmatically generated synthetic videos, transfer effectively to real-world scenarios. We decompose temporal understanding into short-term perceptual primitives (speed, direction) and long-term cognitive primitives, constructing 7.7K CoT and 7K RL samples with ground-truth frame-level annotations through code-based video generation. Despite training on simple geometric shapes, SynRL achieves substantial improvements across 15 benchmarks spanning temporal grounding, complex reasoning, and general video understanding. Remarkably, our 7.7K synthetic CoT samples outperform Video-R1 with 165K real-world samples. We attribute this to fundamental temporal skills, such as tracking frame by frame changes and comparing velocity, that transfer effectively from abstract synthetic patterns to complex real-world scenarios. This establishes a new paradigm for video post-training: video temporal learning through carefully designed synthetic data provides a more cost efficient scaling path.

视频理解合成数据时序建模后训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。