用环境高斯模型提升真实场景反射细节，实现实时高质量新视角合成。
EnvGS: Modeling View-Dependent Appearance with Environment Gaussian
- 用高斯原语显式建模环境反射，融合基础高斯实现全场景外观表示。
- 在真实与合成数据集上均生成更精细的反射，实现实时渲染下的最佳画质。
- 基于GPU光线追踪加速，兼顾高质量重建与实时性能，适合图形应用开发。
从二维图像重建真实场景中的复杂反射对于实现逼真的新视角合成至关重要。现有利用环境图建模远距离光源反射的方法常难以捕捉高频反射细节，且忽略近场反射。本文提出EnvGS，采用一组高斯原语作为显式3D表示来捕捉环境反射。这些环境高斯原语与基础高斯原语结合，用于建模整个场景的外观。为高效渲染环境高斯原语，我们开发了基于光线追踪的渲染器，利用GPU的RT核心实现快速渲染。该方法在保持实时渲染速度的同时，联合优化模型以实现高质量重建。多组真实世界与合成数据集的结果表明，本方法生成的反射细节显著更丰富，在实时新视角合成中达到最佳渲染质量。代码已公开于https://zju3dv.github.io/envgs。
原文摘要 · Abstract (English)
Reconstructing complex reflections in real-world scenes from 2D images is essential for achieving photorealistic novel view synthesis. Existing methods that utilize environment maps to model reflections from distant lighting often struggle with high-frequency reflection details and fail to account for near-field reflections. In this work, we introduce EnvGS, a novel approach that employs a set of Gaussian primitives as an explicit 3D representation for capturing reflections of environments. These environment Gaussian primitives are incorporated with base Gaussian primitives to model the appearance of the whole scene. To efficiently render these environment Gaussian primitives, we developed a ray-tracing-based renderer that leverages the GPU's RT core for fast rendering. This allows us to jointly optimize our model for high-quality reconstruction while maintaining real-time rendering speeds. Results from multiple real-world and synthetic datasets demonstrate that our method produces significantly more detailed reflections, achieving the best rendering quality in real-time novel view synthesis. The code is available at https://zju3dv.github.io/envgs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。