arXiv:2605.30263cs.CV2026-05被引 11

将现有视频生成模型改造为实时可控的交互式世界模型。

minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

论文配图:minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models
图 1 · 摘自论文原文
  • 通过因果强制训练与蒸馏,实现从双向生成到低延迟自回归推理的转化。
  • 在万相2.1和HY1.5模型上验证,支持相机轨迹控制与快速生成。
  • 开源完整流程,适合研究者快速构建或适配交互式视频系统。

近期视频扩散模型在高质量视频生成方面取得显著进展,但将其转化为实时交互式视频世界模型仍具挑战。交互式世界模型需具备可控性、因果性与低延迟推演能力,实际应用中要求涵盖数据构建、可控微调、自回归训练、少步蒸馏与流式推理的全链路支持。本文提出 minWM,一个面向实时交互式视频世界模型的全栈开源框架。该框架可将现有的双向文本到视频(T2V)/图像到视频(TI2V)视频基础模型,转换为可由相机控制的少步自回归世界模型。具体而言,minWM首先对双向视频扩散模型进行相机控制微调,再通过因果强制 / 因果强制++(Causal Forcing / Causal Forcing++)流程,包括自回归扩散训练、因果常微分方程或因果一致性蒸馏、以及非对称DMD,将其蒸馏为少步自回归生成器以实现低延迟推演。框架模块化且架构可扩展:在代表性开源主干模型上实现,包括 Wan2.1-T2V-1.3B 与 HY1.5-TI2V-8B,覆盖基于交叉注意力条件注入与 MMDiT 风格架构。minWM 还支持将现有视频世界模型(如 HY-WorldPlay)适配至新数据分布、训练方案与延迟目标。除发布可运行脚本、检查点、文档与推理代码外,还提供关于相机轨迹质量、可控训练步数与最小批量大小的实用消融实验。我们希望 minWM 成为构建与适配实时交互式视频世界模型的可复现、可扩展范式。项目主页:[https://github.com/shengshu-ai/minWM](https://github.com/shengshu-ai/minWM)

原文摘要 · Abstract (English)

Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time interactive video world models remains challenging. Interactive world models require controllable, causal, and low-latency rollout, which in practice demands a full pipeline spanning data construction, controllable fine-tuning, autoregressive training, few-step distillation, and streaming inference. In this work, we present minWM, a full-stack open-source framework for building real-time interactive video world models. minWM provides an end-to-end pipeline that converts existing bidirectional T2V/TI2V video foundation models into camera-controllable few-step autoregressive world models. Specifically, minWM first fine-tunes a bidirectional video diffusion model with camera control, and then applies the Causal Forcing / Causal Forcing++ pipeline, including AR diffusion training, causal ODE or causal consistency distillation, and asymmetric DMD, to distill it into a few-step autoregressive generator for low-latency rollout. The framework is modular and architecture-extensible: we instantiate it on representative open backbones, including Wan2.1-T2V-1.3B and HY1.5-TI2V-8B, covering both cross-attention-based condition injection and MMDiT-style architectures. minWM also supports adapting existing video world models, such as HY-WorldPlay, to new data distributions, training recipes, and latency targets. Beyond releasing runnable scripts, checkpoints, documentation, and inference code, we provide practical ablations on camera trajectory quality, controllability training steps, and minimal batch-size requirements. We hope minWM serves as a reproducible and extensible recipe for building and adapting real-time interactive video world models. Project Page: [https://github.com/shengshu-ai/minWM](https://github.com/shengshu-ai/minWM)

视频生成扩散模型交互系统开源框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。