用视频生成训练模型推理,让机器像演戏一样一步步思考。
OpenCoF: Learning to Reason Through Video Generation

- 设计新框架OpenCoF,用17000个视频数据训练模型逐步推理。
- 在4个测试中表现优于基线模型,关键提升来自时间序列监督。
- 加入视觉和文本推理标记,让模型更好理解空间与时间逻辑。
推理已成为大模型的核心能力,尤其在需要理解逻辑后果时。近期视频生成模型提供了一种不同于传统链式思维(CoT)的新路径:通过连续帧的时间关联进行推理,称为链式帧(CoF)推理。然而,现有视频生成模型主要基于通用视频语料训练,缺乏多样化的监督与专为CoF推理设计的机制。为此,我们提出OpenCoF框架,包含OpenCoF-17K数据集(涵盖11类任务的17,000个视频)、以及针对CoF推理微调的Wan-CoF模型。在四个视频推理基准上,Wan-CoF相较基线模型Wan2.2-I2V-A14B取得显著提升。进一步探索更先进的设计:引入视觉与文本推理标记,分别捕捉低层视觉线索与高层语义先验,支持空间与时间推理。通过性能对比与注意力分析,我们考察这些标记在模型深度、去噪步数、时空维度上的作用。结果表明,更强的视频推理需同时具备广泛的时间监督与显式的中间推理状态组织机制。我们已开源数据集、模型与代码,以推动面向推理的视频生成研究。
原文摘要 · Abstract (English)
Reasoning has become a core capability for large models, especially when reliable decisions require understanding logical consequences. Recent video generation models offer a reasoning path distinct from previous Chain-of-Thought (CoT): reasoning can unfold through temporally connected frames, known as Chain-of-Frame (CoF) reasoning. However, existing video generators are primarily trained on general video corpora, still lacking diverse supervision and dedicated designs for CoF reasoning. To address this gap, we introduce OpenCoF, a framework comprising the OpenCoF-17K dataset, a reasoning video dataset spanning 11 task families, and Wan-CoF, a fine-tuned video model for studying whether diverse temporal supervision improves CoF behavior. Across four video reasoning benchmarks, Wan-CoF achieves considerable gains over the Wan2.2-I2V-A14B baseline. Building on this, we empirically explore more advanced designs for CoF capabilities, i.e., equipping the model with visual and textual reasoning tokens. This mechanism respectively captures low-level visual cues and high-level semantic priors for spatial and temporal reasoning. Through performance comparisons and attention analysis, we examine how these tokens contribute across model depth, denoising steps, space, and time. Our results suggest that stronger video reasoning requires both broad temporal supervision and explicit mechanisms for organizing intermediate reasoning state. We open-source the dataset, model, and code to facilitate future research on reasoning-oriented video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。