用少视角实现高质量3D渲染，解决高斯点云过拟合问题。
MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views
- 基于多视角先验优化3D高斯点云初始化
- 实时渲染下达到当前最优的视图合成效果
- 适合需要快速重建的3D视觉应用开发者
近年来，神经辐射场（NeRF）的发展推动了少样本新视角合成（NVS）的进步，这是3D视觉应用中的重大挑战。尽管已有诸多尝试降低NeRF对密集输入的需求，其训练与渲染过程仍耗时较长。最近，基于显式点表示的3D高斯喷溅（3DGS）实现了实时高质量渲染。然而，与NeRF类似，3DGS在缺乏约束的情况下容易对训练视角产生过拟合。本文提出MVPGS，一种基于3DGS的少样本NVS方法，通过挖掘多视角先验来改善重建质量。我们利用基于学习的多视角立体（MVS）提升3DGS的几何初始化精度；为缓解过拟合，提出前向映射方法引入符合场景的外观约束；进一步引入视角一致的几何约束以促进高斯参数的合理优化，并采用单目深度正则化作为补充。实验表明，该方法在保持实时渲染速度的同时，达到了当前最优性能。
原文摘要 · Abstract (English)
Recently, the Neural Radiance Field (NeRF) advancement has facilitated few-shot Novel View Synthesis (NVS), which is a significant challenge in 3D vision applications. Despite numerous attempts to reduce the dense input requirement in NeRF, it still suffers from time-consumed training and rendering processes. More recently, 3D Gaussian Splatting (3DGS) achieves real-time high-quality rendering with an explicit point-based representation. However, similar to NeRF, it tends to overfit the train views for lack of constraints. In this paper, we propose \textbf{MVPGS}, a few-shot NVS method that excavates the multi-view priors based on 3D Gaussian Splatting. We leverage the recent learning-based Multi-view Stereo (MVS) to enhance the quality of geometric initialization for 3DGS. To mitigate overfitting, we propose a forward-warping method for additional appearance constraints conforming to scenes based on the computed geometry. Furthermore, we introduce a view-consistent geometry constraint for Gaussian parameters to facilitate proper optimization convergence and utilize a monocular depth regularization as compensation. Experiments show that the proposed method achieves state-of-the-art performance with real-time rendering speed. Project page: https://zezeaaa.github.io/projects/MVPGS/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。