用单步流匹配加速渲染,兼顾速度与真实感。
RenderFlow: Single-Step Neural Rendering via Flow Matching
- 基于流匹配实现端到端单步渲染,无需迭代
- 支持稀疏关键帧引导,提升物理真实性和画质
- 可扩展至逆向渲染任务,适合实时图形应用
传统基于物理的渲染(PBR)依赖复杂的光传输模拟,计算开销大。尽管现有深度学习方法利用扩散模型先验与几何缓冲区(G-buffers)生成视觉逼真的图像,避免显式场景几何和光照模拟,但仍受限于两大问题:扩散过程的迭代性导致显著延迟,且生成模型的随机性影响物理准确性和时间一致性。为此,我们提出一种新的、端到端的、确定性的单步神经渲染框架——RenderFlow,基于流匹配范式构建。为进一步提升渲染质量和泛化能力,我们设计了一种高效且有效的稀疏关键帧引导模块。该方法大幅加速渲染过程,并通过可选地引入稀疏渲染的关键帧作为引导,增强输出的物理合理性与整体视觉质量。所提流水线实现近实时性能,兼具摄影级渲染质量,有效弥合了现代生成模型的效率与传统物理渲染的精度之间的差距。此外,我们还引入一个轻量级适配器模块,可高效复用预训练前向模型,实现内在分解等逆向渲染任务。
原文摘要 · Abstract (English)
Conventional physically based rendering (PBR) pipelines generate photorealistic images through computationally intensive light transport simulations. Although recent deep learning approaches leverage diffusion model priors with geometry buffers (G-buffers) to produce visually compelling results without explicit scene geometry or light simulation, they remain constrained by two major limitations. First, the iterative nature of the diffusion process introduces substantial latency. Second, the inherent stochasticity of these generative models compromises physical accuracy and temporal consistency. In response to these challenges, we propose a novel, end-to-end, deterministic, single-step neural rendering framework, RenderFlow, built upon a flow matching paradigm. To further strengthen both rendering quality and generalization, we propose an efficient and effective module for sparse keyframe guidance. Our method significantly accelerates the rendering process and, by optionally incorporating sparsely rendered keyframes as guidance, enhances both the physical plausibility and overall visual quality of the output. The resulting pipeline achieves near real-time performance with photorealistic rendering quality, effectively bridging the gap between the efficiency of modern generative models and the precision of traditional physically based rendering. Furthermore, we demonstrate the versatility of our framework by introducing a lightweight, adapter-based module that efficiently repurposes the pretrained forward model for the inverse rendering task of intrinsic decomposition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。