提出解耦外观变化的高效方法,提升3D高斯点云渲染质量。
Decoupling Appearance Variations with 3D Consistent Features in Gaussian Splatting
- 在图像层面建模外观变化,不依赖高斯点优化
- 实现跨视角3D一致性,渲染质量达最新水平
- 即插即用,适合各类高斯点云基线
高斯点云已成为新视角合成中的主流3D表示,但仍受多种因素引起的外观变化影响,如现代相机ISP、昼夜差异、天气和局部光照变化。这些变化会导致渲染图像中出现浮影和色彩失真。现有外观建模方法或与渲染过程紧密耦合,阻碍实时性,或仅能处理轻微全局变化,在局部光照变化场景下表现不佳。本文提出DAVIGS,一种即插即用且高效的解耦外观变化方法。通过在图像层面而非高斯点层面转换渲染结果,该方法以极低优化时间和内存开销建模外观变化。同时,利用3D空间中的外观信息对渲染图像进行变换,隐式建立跨视角的3D一致性。我们在多个存在外观变化的场景上验证了该方法,结果表明其在最小训练时间与内存消耗下实现了最先进的渲染质量,且不牺牲渲染速度。此外,该方法可为不同高斯点云基线提供性能提升。
原文摘要 · Abstract (English)
Gaussian Splatting has emerged as a prominent 3D representation in novel view synthesis, but it still suffers from appearance variations, which are caused by various factors, such as modern camera ISPs, different time of day, weather conditions, and local light changes. These variations can lead to floaters and color distortions in the rendered images/videos. Recent appearance modeling approaches in Gaussian Splatting are either tightly coupled with the rendering process, hindering real-time rendering, or they only account for mild global variations, performing poorly in scenes with local light changes. In this paper, we propose DAVIGS, a method that decouples appearance variations in a plug-and-play and efficient manner. By transforming the rendering results at the image level instead of the Gaussian level, our approach can model appearance variations with minimal optimization time and memory overhead. Furthermore, our method gathers appearance-related information in 3D space to transform the rendered images, thus building 3D consistency across views implicitly. We validate our method on several appearance-variant scenes, and demonstrate that it achieves state-of-the-art rendering quality with minimal training time and memory usage, without compromising rendering speeds. Additionally, it provides performance improvements for different Gaussian Splatting baselines in a plug-and-play manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。