用关键帧和运动轨迹引导视频生成,让动作更符合物理规律
CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation

- 通过视觉语言模型分解文本为因果一致的关键帧和物体运动轨迹
- 生成视频时保持高画质,长时序动态更符合物理规律
- 无需训练即可提升复杂场景的生成合理性,适合动态模拟任务
基于扩散模型的视频生成虽在视觉质量与短期时间连贯性上取得进展,但在涉及长期交互的场景中仍难以生成符合物理规律且因果合理的行为。现有方法主要隐式学习物理一致性,而视觉语言模型可显式建模物理法则。本文提出训练免费的CausalMotion框架,通过结构化中间表示显式注入物理推理。核心思路是利用视觉语言模型将文本提示分解为一系列因果一致的关键帧与以物体为中心的运动轨迹,并将其作为软约束对预训练视频扩散模型进行推理阶段引导。该设计在不需额外训练或监督的情况下,显式建模物体动态与因果转变。大量实验表明,本方法在动态密集场景中显著提升物理合理性与时间连贯性,同时保持高感知视频质量。
原文摘要 · Abstract (English)
Recent advances in diffusion-based video generation have significantly improved visual quality and short-term temporal coherence. However, existing methods still struggle to produce videos with physically consistent and causally plausible dynamics, especially in scenarios involving long-horizon interactions. This limitation arises from the fact that video diffusion models primarily learn physical consistency implicitly, while vision-language models can directly model physical laws. Based on this idea, in this work, we propose \textbf{CausalMotion}, a training-free framework that injects explicit physical reasoning into video generation through structured intermediate representations. Our key idea is to decouple reasoning from generation by leveraging a vision-language model to decompose a text prompt into a sequence of causally consistent keyframes and object-centric motion trajectories. These representations are then aligned and integrated as soft constraints to guide a pretrained video diffusion model during inference. This design enables explicit modeling of object dynamics and causal transitions without requiring additional training or supervision. Extensive experiments show that our method consistently improves physical plausibility and temporal coherence, particularly in dynamics-intensive scenarios, while maintaining high perceptual video quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。