arXiv:2607.27380cs.CVcs.AI2026-07被引 5

用可执行代码做思维链,让视频生成更符合物理规律。

VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System

论文配图:VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System
图 1 · 摘自论文原文
  • 用Blender代码作为中间思维链,显式描述场景随时间演化。
  • 在PhyGenBench上提升至0.558,在VBench-2.0上达77.88分。
  • 适合需要可控、可解释视频生成的研究者与开发者。

文本到视频模型虽已具备出色视觉质量,但在生成符合物理规律的动态过程方面仍存在困难,因时间演化需从高度压缩的文本提示中隐式推断。现有思维链方法引入中间计划或视觉状态,但这些表示通常不可执行或时间稀疏,难以完整实现和控制时空过程。为此,我们提出VideoCoCo,一种基于可执行Blender代码的代理双引擎框架。给定文本提示,编码代理生成一段显式定义场景及其时间演化的Blender程序。可执行仿真引擎运行该程序,生成确定性的时空草图,再由生成式视频引擎通过草图引导编辑,转化为逼真视频。此分解将过程推理与高保真视觉实现分离。为适配视频编辑器对模拟草图,我们构建了VideoCoCo-3K数据集,包含3000个草图-指令-目标三元组。VideoCoCo在OmniWeaving基线基础上,使PhyGenBench得分从0.475提升至0.558,VBench-2.0得分从52.18提升至77.88,两个基准均取得最优平均分。结果表明,可执行代码是实现物理一致性视频生成的有效、可控且可检查的中间表示。

原文摘要 · Abstract (English)

Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.

视频生成物理一致性代码思维链双引擎

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。