提升3D高斯点云的光泽与视角依赖外观建模能力
SpecGaussian with Latent Features: A High-quality Modeling of the View-dependent Appearance for 3D Gaussian Splatting
- 用潜空间神经描述符增强每个高斯点的特征表达
- 分离渲染漫反射与镜面反射，视角自适应融合
- 显著改善复杂光照下金属/光滑表面的生成效果
近期，3D高斯点阵（3D-GS）在新视角合成中取得显著进展，实现实时渲染并保证高质量输出。然而，该方法在建模镜面反射及处理各向异性外观成分方面仍存在挑战，尤其在复杂光照条件下难以准确表达视角依赖的颜色变化。此外，3D-GS采用球谐函数表示颜色，难以刻画复杂场景。为此，本文提出Latent-SpecGS，为每个3D高斯点引入通用潜空间神经描述符，更有效地表示包含外观与几何在内的三维特征场。同时设计两个并行卷积网络，分别解码得到漫反射颜色与镜面反射颜色，并通过依赖视角的掩码实现两者的融合，生成最终图像。实验表明，本方法在新视角合成任务中表现优异，显著扩展了3D-GS对具有镜面反射的复杂场景的建模能力。
原文摘要 · Abstract (English)
Recently, the 3D Gaussian Splatting (3D-GS) method has achieved great success in novel view synthesis, providing real-time rendering while ensuring high-quality rendering results. However, this method faces challenges in modeling specular reflections and handling anisotropic appearance components, especially in dealing with view-dependent color under complex lighting conditions. Additionally, 3D-GS uses spherical harmonic to learn the color representation, which has limited ability to represent complex scenes. To overcome these challenges, we introduce Lantent-SpecGS, an approach that utilizes a universal latent neural descriptor within each 3D Gaussian. This enables a more effective representation of 3D feature fields, including appearance and geometry. Moreover, two parallel CNNs are designed to decoder the splatting feature maps into diffuse color and specular color separately. A mask that depends on the viewpoint is learned to merge these two colors, resulting in the final rendered image. Experimental results demonstrate that our method obtains competitive performance in novel view synthesis and extends the ability of 3D-GS to handle intricate scenarios with specular reflections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。