arXiv:2602.00608cs.AIcs.GR2026-02被引 1

通过软硬件协同设计,实现720×480高清实时游戏生成。

Scalable Generative Game Engine: Breaking the Resolution Wall via Hardware-Algorithm Co-Design

  • 分拆世界模型与解码器,分别用算力和内存优化
  • 在720×480分辨率下达到26.4~48.3帧,比之前快50倍
  • 适合追求高画质实时交互的神经游戏开发者

实时生成式游戏引擎代表了交互模拟的新范式,有望用神经世界模型取代传统图形管线。然而现有方法受制于“内存墙”,实际部署仅限低分辨率(如64×64)。本文提出一种可扩展的软硬件协同设计框架,突破分辨率瓶颈。我们发现高分辨率生成存在关键资源错配:世界模型为计算密集型,解码器为内存密集型。为此,设计异构架构,在多颗AI加速器上智能拆分组件。系统包含三项创新:(1) 非对称资源分配策略,优化序列并行下的吞吐;(2) 以内存为中心的操作融合,降低片外带宽;(3) 流形感知的潜在外推机制,利用时间冗余掩盖延迟。在可编程AI加速器集群上验证,实现720×480分辨率下的实时生成,像素吞吐量较基线提升50倍。在连续3D竞速与离散2D平台游戏基准上,分别达到26.4 FPS和48.3 FPS,平均有效延迟仅2.7毫秒。本工作证明,通过架构协同设计破解‘内存墙’不仅是优化,更是实现高保真、低延迟神经游戏的前提。

原文摘要 · Abstract (English)

Real-time generative game engines represent a paradigm shift in interactive simulation, promising to replace traditional graphics pipelines with neural world models. However, existing approaches are fundamentally constrained by the ``Memory Wall,'' restricting practical deployments to low resolutions (e.g., $64 \times 64$). This paper bridges the gap between generative models and high-resolution neural simulations by introducing a scalable \textit{Hardware-Algorithm Co-Design} framework. We identify that high-resolution generation suffers from a critical resource mismatch: the World Model is compute-bound while the Decoder is memory-bound. To address this, we propose a heterogeneous architecture that intelligently decouples these components across a cluster of AI accelerators. Our system features three core innovations: (1) an asymmetric resource allocation strategy that optimizes throughput under sequence parallelism constraints; (2) a memory-centric operator fusion scheme that minimizes off-chip bandwidth usage; and (3) a manifold-aware latent extrapolation mechanism that exploits temporal redundancy to mask latency. We validate our approach on a cluster of programmable AI accelerators, enabling real-time generation at $720 \times 480$ resolution -- a $50\times$ increase in pixel throughput over prior baselines. Evaluated on both continuous 3D racing and discrete 2D platformer benchmarks, our system delivers fluid 26.4 FPS and 48.3 FPS respectively, with an amortized effective latency of 2.7 ms. This work demonstrates that resolving the ``Memory Wall'' via architectural co-design is not merely an optimization, but a prerequisite for enabling high-fidelity, responsive neural gameplay.

生成游戏软硬件协同高分辨率生成神经渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。