用多视角一致性提升高斯点云反射材质建模精度
MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference
- 通过多视角约束优化高斯点的材质映射
- 实现更逼真的镜面反射与间接光照效果
- 适合需要高保真渲染的3D重建与视觉生成任务
从2D图像建模反射对于真实感渲染和新视角合成至关重要。现有方法在高斯原语中引入反射相关的材质属性以支持基于物理的渲染(PBR)与高斯点云渲染,但材质推断常因约束不足,尤其在环境建模有限时,导致光照伪影并降低泛化能力。本文从多视角角度重新审视该问题,表明多视角一致的材质推断与更物理化的环境建模是实现高斯点云准确反射的关键。为此,我们在延迟着色阶段强制2D高斯点生成多视角一致的材质图;同时追踪跨视角的光度变化,识别高反射区域,作为反射强度项的强先验。为处理由物体间遮挡引起的间接光照,我们提出基于2DGS的射线追踪环境建模策略,实现间接辐射的逼真渲染。在多个主流基准上的实验表明,本方法能准确恢复光照与几何信息,在新视角合成中达到当前最优渲染质量。
原文摘要 · Abstract (English)
Modeling reflections from 2D images is essential for photorealistic rendering and novel view synthesis. Recent approaches enhance Gaussian primitives with reflection-related material attributes to enable physically based rendering (PBR) with Gaussian Splatting. However, the material inference often lacks sufficient constraints, especially under limited environment modeling, resulting in illumination aliasing and reduced generalization. In this work, we revisit the problem from a multi-view perspective and show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting. To this end, we enforce 2D Gaussians to produce multi-view consistent material maps during deferred shading. We also track photometric variations across views to identify highly reflective regions, which serve as strong priors for reflection strength terms. To handle indirect illumination caused by inter-object occlusions, we further introduce an environment modeling strategy through ray tracing with 2DGS, enabling photorealistic rendering of indirect radiance. Experiments on widely used benchmarks show that our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。