通过自集成提升稀疏视图下的3D高斯点云渲染质量
Self-Ensembling Gaussian Splatting for Few-Shot Novel View Synthesis
- 用不确定性引导的动态扰动策略实现模型自集成
- 在少样本条件下显著提升新视角合成效果，超越现有方法
- 适合需要高效、鲁棒3D重建的视觉研究者与工程师
3D高斯点云（3DGS）在新视角合成（NVS）中表现优异，但在稀疏视图训练时易过拟合，限制了泛化能力。本文提出自集成高斯点云（SE-GS），通过引入不确定性感知的扰动策略实现自集成。联合训练一个Δ-模型和Σ-模型，Δ-模型在训练过程中根据渲染不确定性动态扰动，生成多样化的扰动模型，计算开销极低。通过最小化Σ-模型与各扰动模型间的差异，构建出稳健的3DGS模型集合，推理时使用Σ-模型生成新视角图像。在LLFF、Mip-NeRF360、DTU和MVImgNet数据集上的实验表明，该方法在少样本条件下显著提升NVS质量，优于当前最优方法。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has demonstrated remarkable effectiveness in novel view synthesis (NVS). However, 3DGS tends to overfit when trained with sparse views, limiting its generalization to novel viewpoints. In this paper, we address this overfitting issue by introducing Self-Ensembling Gaussian Splatting (SE-GS). We achieve self-ensembling by incorporating an uncertainty-aware perturbation strategy during training. A $\mathbfΔ$-model and a $\mathbfΣ$-model are jointly trained on the available images. The $\mathbfΔ$-model is dynamically perturbed based on rendering uncertainty across training steps, generating diverse perturbed models with negligible computational overhead. Discrepancies between the $\mathbfΣ$-model and these perturbed models are minimized throughout training, forming a robust ensemble of 3DGS models. This ensemble, represented by the $\mathbfΣ$-model, is then used to generate novel-view images during inference. Experimental results on the LLFF, Mip-NeRF360, DTU, and MVImgNet datasets demonstrate that our approach enhances NVS quality under few-shot training conditions, outperforming existing state-of-the-art methods. The code is released at: https://sailor-z.github.io/projects/SEGS.html.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。