MoWorld实现50帧实时交互,无需高端显卡即可运行。
MoWorld: A Flash World Model

- 基于3D原生数据引擎构建几何一致训练数据
- 支持50帧每秒实时推理,成本仅为现有模型30%-50%
- 首个在NPU上实现的实时世界模型,适合大规模部署
世界模型的未来不仅依赖于模型能力的扩展,更需提升实用性和推理效率。高帧率推理可实现实时感知、规划与控制,适用于真实世界的自主系统。为此,我们提出低成本但高性能的闪速世界模型MoWorld,其端到端框架涵盖数据生成、预训练、蒸馏与高效推理,可在无高端GPU情况下实现高达50 FPS的实时交互,并保持电影级画质。为实现大规模真实世界部署,MoWorld在整个开发流程中联合优化模型能力与成本。不同于依赖大规模视频语料的方法,MoWorld基于我们自建的大规模3D视觉与生成建模流水线积累的3D原生数据引擎,高效构建跨真实与合成环境的几何一致性训练数据。在此基础上,采用课程化跨帧预训练策略实现稳定可扩展的世界模型学习,提出高效的去噪步蒸馏算法降低扩散训练成本,并设计混合精度并行推理框架实现低成本实时部署。MoWorld是首个基于神经处理单元(NPU)构建的实时交互世界模型,在该设备上可达50 FPS,为世界模型的大规模实际应用提供可行基础。综合评估表明,其平均推理成本仅为现有模型的30%-50%,并展示了多样化的应用场景。
原文摘要 · Abstract (English)
The future of World Models depends not only on scaling model capability, but also on scaling practicality and inference efficiency. High-frame-rate inference enables responsive perception, planning, and control in real-world autonomous systems. To this end, we present MoWorld, a cost-effective yet high-performance Flash World Model with an end-to-end framework spanning data generation, pre-training, distillation, and efficient inference, enabling up to 50 FPS real-time interaction with cinematic visual quality without the need of high-end GPUs. To enable large-scale real-world deployment, MoWorld jointly optimizes model capability and cost throughout the entire development pipeline. Specifically, unlike existing approaches that primarily rely on large-scale video corpora, MoWorld is built upon a scalable 3D-native data engine accumulated from our large-scale 3D vision and generative modeling pipeline, enabling the efficient construction of geometrically consistent training data across diverse real-world and synthetic environments. Based on this foundation, a curriculum cross-frame pre-training strategy for stable and scalable World Model learning, an efficient denoising-step distillation algorithm to reduce diffusion training cost, and a mixed-precision parallel inference framework for low-cost real-time deployment. MoWorld is the first real-time interactive World Model built on the Neural Processing Unit (NPU) and can achieves up to 50 FPS in such the devices, enabling practical and efficient deployment at scale. Comprehensive evaluations demonstrate that MoWorld achieves leading performance; notably, its average inference cost is only 30\%-50\% of that of existing World Models, providing a practical foundation for large-scale real-world applications of World Models. We also demonstrate diverse applications of MoWorld.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。