通过谱熵分析改进3D高斯点云,消除尖刺伪影并提升细节表现。
Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy
- 引入谱熵感知的分割策略,让高斯点能自适应调整形状。
- 在1080p分辨率下渲染时,显著减少针状伪影,保持视图一致性。
- 适合追求高质量三维重建与真实感渲染的研究者和开发者。
最近,3D高斯点云(3D-GS)在新视角合成中取得了出色效果,表现出高保真度和高效性。然而,其在提高采样率时容易产生针状伪影。尽管Mip-Splatting通过3D平滑滤波器和2D Mip滤波器缓解了频率问题,但仍导致过度模糊,且针状高斯仍存在。我们对优化与稠密化过程中的协方差矩阵进行谱分析发现,现有3D-GS缺乏形状感知能力,仅依赖谱半径和视角位置梯度决定分裂,导致小梯度、低谱熵的针状高斯无法分裂,从而过拟合高频细节。此外,3D-GS与Mip-Splatting使用的滤波器在缩放时降低谱熵并增加条件数,引发视图不一致和更明显的伪影。我们的Spectral-GS基于谱分析,提出3D形状感知分裂与2D视图一致滤波策略,有效解决上述问题,显著提升3D-GS对高频细节的表达能力,实现无明显伪影的高质量逼真渲染。
原文摘要 · Abstract (English)
Recently, 3D Gaussian Splatting (3D-GS) has achieved impressive results in novel view synthesis, demonstrating high fidelity and efficiency. However, it easily exhibits needle-like artifacts, especially when increasing the sampling rate. Mip-Splatting tries to remove these artifacts with a 3D smoothing filter for frequency constraints and a 2D Mip filter for approximated supersampling. Unfortunately, it tends to produce over-blurred results, and sometimes needle-like Gaussians still persist. Our spectral analysis of the covariance matrix during optimization and densification reveals that current 3D-GS lacks shape awareness, relying instead on spectral radius and view positional gradients to determine splitting. As a result, needle-like Gaussians with small positional gradients and low spectral entropy fail to split and overfit high-frequency details. Furthermore, both the filters used in 3D-GS and Mip-Splatting reduce the spectral entropy and increase the condition number during zooming in to synthesize novel view, causing view inconsistencies and more pronounced artifacts. Our Spectral-GS, based on spectral analysis, introduces 3D shape-aware splitting and 2D view-consistent filtering strategies, effectively addressing these issues, enhancing 3D-GS's capability to represent high-frequency details without noticeable artifacts, and achieving high-quality photorealistic rendering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。