通过生成验证图控制高斯点数,缓解稀疏视角3DGS过拟合问题。
VGNC: Reducing the Overfitting of Sparse-view 3DGS via Validation-guided Gaussian Number Control
- 用生成式新视角合成模型生成验证图像
- 根据验证图像自动调节最优高斯点数,减少过拟合
- 提升渲染质量同时降低点数、存储与计算开销
稀疏视角三维重建是实际应用中的基础但极具挑战的任务。近年来,基于3D高斯溅射(3DGS)框架的方法在该任务上取得显著进展,但仍存在严重过拟合问题。为缓解此问题,我们提出一种基于生成式新视角合成(NVS)模型的验证引导高斯点数控制方法(VGNC)。据我们所知,这是首个利用生成验证图像来缓解稀疏视角3DGS过拟合的尝试。具体而言,首先设计一种基于生成式NVS模型的验证图像生成方法;随后提出一种高斯点数调控策略,利用生成的验证图像确定最优高斯数量,从而减轻过拟合。我们在多个稀疏视角3DGS基线和数据集上进行了详尽实验,结果表明:该方法不仅有效抑制过拟合,还提升了测试集上的渲染质量,同时减少了高斯点数。这一改进降低了存储需求,并加速了训练与渲染过程。代码将公开。
原文摘要 · Abstract (English)
Sparse-view 3D reconstruction is a fundamental yet challenging task in practical 3D reconstruction applications. Recently, many methods based on the 3D Gaussian Splatting (3DGS) framework have been proposed to address sparse-view 3D reconstruction. Although these methods have made considerable advancements, they still show significant issues with overfitting. To reduce the overfitting, we introduce VGNC, a novel Validation-guided Gaussian Number Control (VGNC) approach based on generative novel view synthesis (NVS) models. To the best of our knowledge, this is the first attempt to alleviate the overfitting issue of sparse-view 3DGS with generative validation images. Specifically, we first introduce a validation image generation method based on a generative NVS model. We then propose a Gaussian number control strategy that utilizes generated validation images to determine the optimal Gaussian numbers, thereby reducing the issue of overfitting. We conducted detailed experiments on various sparse-view 3DGS baselines and datasets to evaluate the effectiveness of VGNC. Extensive experiments show that our approach not only reduces overfitting but also improves rendering quality on the test set while decreasing the number of Gaussian points. This reduction lowers storage demands and accelerates both training and rendering. The code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。