让游戏世界实时生成画面,速度达30帧/秒。
Generative World Renderer at the Speed of Play

- 将渲染模型改造成多步自回归流式处理,实现低延迟生成。
- 从0.56帧提升至31.54帧,满足实时交互需求。
- 适合游戏开发与可控制的动态场景生成应用。
生成式世界渲染器AlayaRenderer接收物理引擎导出的结构化世界状态,合成RGB画面。与基于文本或控制提示生成图像的模型不同,AlayaRenderer保持场景结构不变,不干扰底层物理动态,为交互式世界建模和用户可控游戏提供了新路径。然而原版AlayaRenderer计算开销过大,难以实时部署。本技术报告提出AlayaRenderer-Flash,一种面向实时的生成式前向渲染器,将帧率从0.56 FPS提升至31.54 FPS,达到游戏播放速度。该模型将原渲染器重构为多步自回归流式模型,并引入轻量级蒸馏编码器,实现高效潜在编码与画面重建。在保留教师模型的G-buffer与文本提示接口的同时,支持对无限长度输入流的连续渲染。我们在多个G-buffer数据流上评估了内容保真度、时间一致性、跨窗口稳定性、提示可控性及运行效率。结果表明,AlayaRenderer-Flash显著降低推理成本,同时保持教师模型的核心渲染能力。通过与物理引擎集成,构建了运行在30 FPS的完整可玩生成世界。
原文摘要 · Abstract (English)
Generative world renderer AlayaRenderer receives structured world states exported from physics engines and synthesizes RGB frames. Unlike models that generate frames from text/control-hints prompts, AlayaRenderer preserves scene structure without altering the underlying world dynamics. This demonstrates an alternative path toward interactive world modeling and user-controllable play. However, the original AlayaRenderer is too computationally expensive for real-time deployment. This technical report introduces AlayaRenderer-Flash, a real-time-oriented generative forward world renderer that pushes AlayaRenderer from 0.56 FPS to 31.54 FPS, reaching the speed of play. AlayaRenderer-Flash reformulates the original renderer as a few-step autoregressive streaming model and introduces lightweight distilled codecs for efficient latent encoding and frame reconstruction. It retains the teacher model's G-buffer and text-prompt interfaces while enabling continuous rendering over input streams of unbounded length. We evaluate AlayaRenderer-Flash on G-buffer streams across content preservation, temporal consistency, cross-window stability, prompt controllability, and runtime efficiency. Our results show that AlayaRenderer-Flash substantially reduces inference cost while preserving the core rendering capabilities of the teacher model. By integrating AlayaRenderer-Flash with a physics engine, we build a fully playable generative world running at 30 FPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。