用可变材质的高斯点提升图像逆渲染,实现更真实的光照重演。
SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering
- 引入空间可变高斯点,每个点支持变化的材质和法向参数。
- 在光照重演任务中比现有方法提升3.5 dB,新视角合成质量高出2.5 dB。
- 适合做高质量3D重建与真实感光照编辑的研究者和开发者。
从图像中重建3D资产(即逆渲染)因其病态性仍具挑战。3D高斯点云(3DGS)在新视角合成任务中表现优异,但现有方法将辐射度分解为材质和光照参数时,因每个高斯点材质恒定、法向固定且缺乏间接光照物理约束,导致重光效果差、出现伪影和不自然的间接光照。本文提出空间可变高斯逆渲染(SVG-IR),通过引入空间可变高斯(SVG)表示,使每个高斯点具备空间变化的材质与法向参数,并设计类似图形管线中顶点/片段着色的点云渲染方案。同时融合基于物理的间接光照模型,显著提升真实感。SVG-IR在新视角合成上比先进NeRF方法提升2.5 dB PSNR,重光任务中优于现有高斯方法3.5 dB,且保持实时渲染速度。
原文摘要 · Abstract (English)
Reconstructing 3D assets from images, known as inverse rendering (IR), remains a challenging task due to its ill-posed nature. 3D Gaussian Splatting (3DGS) has demonstrated impressive capabilities for novel view synthesis (NVS) tasks. Methods apply it to relighting by separating radiance into BRDF parameters and lighting, yet produce inferior relighting quality with artifacts and unnatural indirect illumination due to the limited capability of each Gaussian, which has constant material parameters and normal, alongside the absence of physical constraints for indirect lighting. In this paper, we present a novel framework called Spatially-vayring Gaussian Inverse Rendering (SVG-IR), aimed at enhancing both NVS and relighting quality. To this end, we propose a new representation-Spatially-varying Gaussian (SVG)-that allows per-Gaussian spatially varying parameters. This enhanced representation is complemented by a SVG splatting scheme akin to vertex/fragment shading in traditional graphics pipelines. Furthermore, we integrate a physically-based indirect lighting model, enabling more realistic relighting. The proposed SVG-IR framework significantly improves rendering quality, outperforming state-of-the-art NeRF-based methods by 2.5 dB in peak signal-to-noise ratio (PSNR) and surpassing existing Gaussian-based techniques by 3.5 dB in relighting tasks, all while maintaining a real-time rendering speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。