通过多片段预测加速视频生成训练与推理,提升高帧率下的准确性。
Next Forcing: Causal World Modeling with Multi-Chunk Prediction

- 引入多片段预测框架,用未来多个时间步的监督信号增强模型
- 在50帧/秒下训练速度提升2.3倍,准确率比现有方法高93.1%
- 推理时可并行生成下一帧,实现2倍加速,适合高帧率视频建模
自回归视频生成已成为世界动作模型(WAMs)的强大范式。然而,现有方法在高帧率下存在训练收敛慢、精度有限的问题,因训练仅依赖当前片段而缺乏对未来动态的显式信号;同时推理也因迭代去噪过程缓慢。本文提出Next Forcing,一种基于多片段预测(MCP)的因果世界建模框架,实现更快训练、更高精度和加速推理。受大语言模型多标记预测启发,Next Forcing引入MCP训练目标,通过轻量级辅助模块在多个未来时间步(next¹, next², next³)同时去噪视频片段。这些模块构成跨预测深度的因果链,融合主模型多层中间特征,使近未来预测指导远未来预测,并向主模型提供密集的多尺度时序监督。训练中,该框架显著加速收敛并提升精度:在50 fps下,于5000步训练时相比LingBot-VA相对提升93.1%,收敛速度快2.3倍,在RoboTwin基准上取得94.1%(Clean)/93.5%(Random)新纪录。推理阶段,保留MCP模块可并行预测下一视频片段,实现2倍加速。该方法还在PhyWorld(物理规律遵循性)上表现优异,通用视频预训练中FVD降低超50%。
原文摘要 · Abstract (English)
Autoregressive video generation has emerged as a powerful paradigm for World Action Models (WAMs). However, existing approaches suffer from slow training convergence and limited converged accuracy, particularly at high frame rates, as the training supervision is confined to the current chunk without explicit signals about future dynamics; they also suffer from slow inference due to iterative video denoising. In this paper, we present Next Forcing, a multi-chunk prediction (MCP) framework for causal world modeling that enables faster training, higher accuracy, and accelerated inference. Inspired by multi-token prediction in large language models, Next Forcing introduces an MCP training objective that augments the main model with lightweight auxiliary MCP modules to simultaneously denoise video chunks at multiple future temporal horizons (next$^1$, next$^2$, next$^3$ chunks). These MCP modules form a causal chain across prediction depths, where intermediate features fused from multiple layers of the main model are leveraged to predict future dynamics, allowing near-future predictions to inform farther-future ones and providing dense multi-scale temporal supervision back to the main model. During training, the MCP modules significantly accelerate convergence and improve converged accuracy, especially at high frame rates: at 50 fps, Next Forcing achieves a 93.1% relative improvement over LingBot-VA at 5k training steps and 2.3x faster convergence, and establishes new state-of-the-art results on the RoboTwin benchmark (94.1/93.5% on Clean/Random). At inference, the MCP modules can be retained to predict the next video chunk in parallel with the current one, achieving 2x inference acceleration. Next Forcing also demonstrates significant improvements on PhyWorld, a benchmark evaluating adherence to physical laws in video generation, and over 50% FVD reduction on general video pretraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。