用辐射场生成新视角,提升稀疏视图下高斯点云渲染质量。
AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings

- 用辐射场合成额外视角图像,增强优化监督信号。
- 在稀疏视图mip-NeRF 360数据集上,重建质量显著优于标准高斯点云方法。
- 支持实时渲染,适合稀疏视角3D重建任务的科研与工程应用。
在稀疏视图场景中实现实时高质量新视角生成仍是3D视觉的核心挑战。神经辐射场虽能从有限观测中实现鲁棒重建,但依赖体素渲染导致计算开销大、推理慢。相比之下,高斯点云渲染通过光栅化实现实时渲染,但其优化对初始几何质量极为敏感。在稀疏视图下,有限观测常导致几何不完整或噪声严重。本文提出AugSplat,一种基于辐射场视图增强的高斯点云优化框架。首先在稀疏输入视图上训练辐射场,并利用其合成邻近新视角的图像,扩大可用监督的视图空间覆盖。这些合成图像作为辅助监督用于高斯点云优化。提出两种变体:Staged AugSplat在初期使用合成视图优化,随后切换至真实图像;Dual AugSplat则联合优化真实与合成图像,合成损失权重逐渐衰减。在稀疏视图mip-NeRF 360数据集上的实验表明,AugSplat显著提升重建质量。Staged AugSplat表现最佳,Dual AugSplat性能相近且全程保留真实图像监督,两者均保持推理阶段实时性。
原文摘要 · Abstract (English)
Generating high-quality novel views at real-time frame rates remains a central challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric rendering leads to high computational cost and slow inference. In contrast, Gaussian Splatting methods achieve real-time rendering through rasterization, but their optimization is highly sensitive to the quality of the initial geometry. This sensitivity becomes especially problematic in sparse-view settings, where limited observations often lead to incomplete or noisy point-cloud reconstructions. In this work, we present AugSplat, a simple framework for improving Gaussian Splatting in sparse-view regimes using radiance-field-based view augmentation. We first train a radiance field on the sparse input views and use it to synthesize additional images from nearby novel viewpoints, increasing the effective view-space coverage available for supervision. These synthetic views are then used as auxiliary supervision during Gaussian Splatting optimization. We study two variants: Staged AugSplat, which uses synthetic views for an initial optimization phase before switching to real images, and Dual AugSplat, which jointly trains on real and synthetic views with a decaying synthetic loss weight. Experiments on sparse-view mip-NeRF 360 scenes show that AugSplat improves reconstruction quality over standard Gaussian Splatting. Staged AugSplat achieves the strongest average performance, while Dual AugSplat provides a closely performing formulation that keeps real-image supervision active throughout training, and both variants preserve real-time rendering at inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。