arXiv:2512.16670cs.CVcs.GR2025-12被引 3

用G-buffer和前一帧生成连贯的逼真画面,适合实时交互场景。

FrameDiffuser: G-Buffer-Conditioned Diffusion for Neural Forward Frame Rendering

  • 基于G-buffer和自身前一帧进行自回归生成,保证时间一致性。
  • 在数百到上千帧上保持稳定渲染,速度满足消费级游戏设备需求。
  • 专精于特定环境训练,光照阴影反射更真实,适合影视级实时渲染。

交互式应用中的神经渲染需要将几何与材质属性(G-buffer)转化为具有真实光照的逐帧图像。现有基于扩散模型的方法存在局限:单帧模型如RGBX生成独立帧,缺乏时序一致性;视频模型如DiffusionRenderer计算开销大,需完整序列输入,难以用于依赖用户输入的实时场景。本文提出FrameDiffuser,一种自回归神经渲染框架,通过条件化于G-buffer数据和自身先前输出,实现时序一致、逼真的帧生成。初始帧后,仅依赖实时传入的G-buffer(含几何、材质、表面属性),结合前一帧提供时序引导,可稳定生成数百至数千帧。其双条件架构融合ControlNet结构引导与ControlLoRA时序一致性控制,采用三阶段训练策略确保自回归生成稳定性。模型针对特定环境进行专精训练,优先保障一致性与推理速度,相比通用方法显著提升光照、阴影与反射的真实感,实现高质量实时渲染。

原文摘要 · Abstract (English)

Neural rendering for interactive applications requires translating geometric and material properties (G-buffer) to photorealistic images with realistic lighting on a frame-by-frame basis. While recent diffusion-based approaches show promise for G-buffer-conditioned image synthesis, they face critical limitations: single-image models like RGBX generate frames independently without temporal consistency, while video models like DiffusionRenderer are too computationally expensive for most consumer gaming sets ups and require complete sequences upfront, making them unsuitable for interactive applications where future frames depend on user input. We introduce FrameDiffuser, an autoregressive neural rendering framework that generates temporally consistent, photorealistic frames by conditioning on G-buffer data and the models own previous output. After an initial frame, FrameDiffuser operates purely on incoming G-buffer data, comprising geometry, materials, and surface properties, while using its previously generated frame for temporal guidance, maintaining stable, temporal consistent generation over hundreds to thousands of frames. Our dual-conditioning architecture combines ControlNet for structural guidance with ControlLoRA for temporal coherence. A three-stage training strategy enables stable autoregressive generation. We specialize our model to individual environments, prioritizing consistency and inference speed over broad generalization, demonstrating that environment-specific training achieves superior photorealistic quality with accurate lighting, shadows, and reflections compared to generalized approaches.

神经渲染扩散模型实时生成时序一致

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。