用深度先验提升少视角3D高斯点云渲染质量
Few-shot Novel View Synthesis using Depth Aware 3D Gaussian Splatting
- 引入单目深度预测与尺度不变损失约束形状
- 用低阶球谐函数建模颜色防过拟合
- 保留所有点云避免稀疏化，适合少图场景
3D高斯点云已超越神经辐射场，在新视角合成中实现更低计算成本和实时高质量渲染。然而在仅有少量输入视图时，其性能显著下降。本文提出一种面向少样本新视角合成的深度感知高斯点云方法。利用单目深度预测作为先验，并结合尺度不变深度损失，仅在少数输入视图下约束三维形状。同时采用低阶球谐函数建模颜色，避免过拟合。此外，发现原方法定期移除低透明度点会导致点云过于稀疏，影响重建质量，因此本文保留全部点云，从而在少视图条件下获得更好重建效果。实验表明，相比传统3D高斯点云方法，本方法在峰值信噪比上提升10.5%，结构相似性提升6%，感知相似性提升14.1%，验证了方法的有效性。
原文摘要 · Abstract (English)
3D Gaussian splatting has surpassed neural radiance field methods in novel view synthesis by achieving lower computational costs and real-time high-quality rendering. Although it produces a high-quality rendering with a lot of input views, its performance drops significantly when only a few views are available. In this work, we address this by proposing a depth-aware Gaussian splatting method for few-shot novel view synthesis. We use monocular depth prediction as a prior, along with a scale-invariant depth loss, to constrain the 3D shape under just a few input views. We also model color using lower-order spherical harmonics to avoid overfitting. Further, we observe that removing splats with lower opacity periodically, as performed in the original work, leads to a very sparse point cloud and, hence, a lower-quality rendering. To mitigate this, we retain all the splats, leading to a better reconstruction in a few view settings. Experimental results show that our method outperforms the traditional 3D Gaussian splatting methods by achieving improvements of 10.5% in peak signal-to-noise ratio, 6% in structural similarity index, and 14.1% in perceptual similarity, thereby validating the effectiveness of our approach. The code will be made available at: https://github.com/raja-kumar/depth-aware-3DGS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。