arXiv:2608.01127cs.CV2026-08

用少量算力从零训练可流式生成视频的世界模型。

MiniWorld: Democratizing the Training of Video World Models from Scratch

论文配图:MiniWorld: Democratizing the Training of Video World Models from Scratch
图 1 · 摘自论文原文
  • 用潜在空间扩散模型+分块渐增噪声调度,实现端到端训练。
  • 单8卡服务器几天内完成训练,推理支持低延迟流式生成。
  • 开源代码和权重,推动视频世界模型研究普及化。

视频世界模型基于历史观测与控制信号预测未来,通过自回归状态转移实现长时序生成。与仅捕捉视觉外观和运动的传统视频生成模型不同,视频世界模型学习代理行为下的环境演化动态,为具身智能和交互式仿真提供基础。近期进展多依赖预训练视频生成模型的后训练或知识蒸馏,但这类方法常需复杂流程、大量算力,且存在双向预训练与因果推理之间的不匹配问题。已有研究表明,从零开始训练自回归视频世界模型可行且可扩展。然而,社区仍缺乏轻量、透明、完全可复现且仅需小规模算力即可端到端训练的基线框架。我们提出 MiniWorld,一个可复现的从零训练流式视频世界模型的框架。MiniWorld 在预训练视频变分自编码器(Video VAE)的潜在空间中,采用块因果视频扩散变换器,并通过流匹配进行训练。基于扩散强迫机制,其引入分块非递减噪声调度与两阶段持续训练,以提升时间建模能力与训练稳定性。推理时,结合滚动键值缓存与流水线异步去噪,实现在有限计算下的高效流式生成。整个模型可在单台8卡服务器上数日内完成训练。通过发布训练与推理代码库及预训练检查点,我们希望 MiniWorld 能促进视频世界建模领域的后续研究。

原文摘要 · Abstract (English)

Video world models predict future observations conditioned on historical observations and control signals, enabling long-horizon generation through autoregressive state transitions. Unlike conventional video generation models that primarily capture visual appearance and motion, video world models learn the underlying dynamics governing environment evolution under agent actions, providing a foundation for embodied AI and interactive simulation. Recent progress has largely relied on adapting pretrained video generation models through post-training or distillation. Although effective, these approaches often require complex training pipelines, substantial computational resources, and suffer from the mismatch between bidirectional pretraining and causal streaming inference. Recent studies have shown that training autoregressive video world models from scratch is feasible and scalable. However, the community still lacks a lightweight, transparent, and fully reproducible baseline trainable end-to-end with modest computational resources. We present MiniWorld, a reproducible framework for training streaming video world models from scratch. MiniWorld employs a block-causal Video Diffusion Transformer trained with Flow Matching in the latent space of a pretrained Video VAE. Building on Diffusion Forcing, it adopts a chunk-wise non-decreasing noise schedule and two-stage continued training to improve temporal modeling and stability. During inference, MiniWorld combines a rolling KV cache with pipelined asynchronous denoising for efficient streaming generation under bounded computation. The entire model can be trained within several days on a single 8-GPU server. By releasing the training and inference codebase and pretrained checkpoints, we hope MiniWorld will facilitate future research on video world modeling.

视频世界模型扩散模型流式生成轻量化训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。