arXiv:2511.17450cs.CVcs.AI2025-11被引 3

用草图验证提升视频生成的物理合理性与运动连贯性

Planning with Sketch-Guided Verification for Physics-Aware Video Generation

  • 通过草图验证循环在生成前筛选更符合物理规律的运动轨迹
  • 在两个基准上显著提升运动质量与长期一致性,效率远超迭代生成方法
  • 无需训练,适合需要高真实感视频生成的研究与应用

近期视频生成方法越来越多依赖于规划中间控制信号(如物体轨迹)以提升时间连贯性和运动保真度。然而,这些方法多采用单次规划,仅限于简单运动;或采用迭代精炼,需多次调用生成器,计算开销大。为此,我们提出SketchVerify——一种无需训练、基于草图验证的规划框架,通过测试时采样与验证循环,在全视频生成前提升运动规划质量,实现更动态一致的轨迹(即物理合理且指令一致的运动)。给定提示和参考图像,该方法预测多个候选运动计划,并利用视觉-语言验证器联合评估其与指令的语义对齐度和物理合理性。为高效评分,将每条轨迹渲染为轻量级视频草图(对象叠加于静态背景),避免昂贵的重复扩散合成,性能相当。迭代优化直至找到满意方案,再交由轨迹条件生成器完成最终合成。在WorldModelBench和PhyWorldBench上的实验表明,该方法显著优于竞争基线,在运动质量、物理真实性和长期一致性方面均有提升,且效率大幅提高。消融研究进一步显示,增加轨迹候选数量可持续提升整体性能。

原文摘要 · Abstract (English)

Recent video generation approaches increasingly rely on planning intermediate control signals such as object trajectories to improve temporal coherence and motion fidelity. However, these methods mostly employ single-shot plans that are typically limited to simple motions, or iterative refinement which requires multiple calls to the video generator, incuring high computational cost. To overcome these limitations, we propose SketchVerify, a training-free, sketch-verification-based planning framework that improves motion planning quality with more dynamically coherent trajectories (i.e., physically plausible and instruction-consistent motions) prior to full video generation by introducing a test-time sampling and verification loop. Given a prompt and a reference image, our method predicts multiple candidate motion plans and ranks them using a vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility. To efficiently score candidate motion plans, we render each trajectory as a lightweight video sketch by compositing objects over a static background, which bypasses the need for expensive, repeated diffusion-based synthesis while achieving comparable performance. We iteratively refine the motion plan until a satisfactory one is identified, which is then passed to the trajectory-conditioned generator for final synthesis. Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality, physical realism, and long-term consistency compared to competitive baselines while being substantially more efficient. Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance.

视频生成物理模拟运动规划草图验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。