无需外部先验，利用双目一致性实现稀疏视图的高质量3D重建。
Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesis
- 通过双目图像间视差引导的图像对齐，自监督学习视图一致性。
- 引入高斯不透明度约束，提升3D高斯点分布的鲁棒性与效率。
- 适用于稀疏视角输入，适合需要高精度3D重建的场景。
从稀疏输入中进行新视角合成是3D计算机视觉中的关键但具挑战性任务。现有方法在3D高斯点云渲染中引入神经先验（如深度先验）作为额外监督，相比基于NeRF的方法展现出更优的质量与效率。然而，来自2D预训练模型的神经先验通常噪声大且模糊，难以精确指导辐射场的学习。本文提出一种无需外部先验的新方法，仅依赖双目图像对间的立体一致性进行自监督学习。具体而言，我们利用视差引导的图像扭曲构建每对双目图像，并从中提取自监督信号。同时，引入高斯不透明度约束，规范高斯点位置分布，避免冗余，提升从稀疏视角推断3D高斯点的鲁棒性与效率。在LLFF、DTU和Blender数据集上的大量实验表明，本方法显著优于当前最优方法。
原文摘要 · Abstract (English)
Novel view synthesis from sparse inputs is a vital yet challenging task in 3D computer vision. Previous methods explore 3D Gaussian Splatting with neural priors (e.g. depth priors) as an additional supervision, demonstrating promising quality and efficiency compared to the NeRF based methods. However, the neural priors from 2D pretrained models are often noisy and blurry, which struggle to precisely guide the learning of radiance fields. In this paper, We propose a novel method for synthesizing novel views from sparse views with Gaussian Splatting that does not require external prior as supervision. Our key idea lies in exploring the self-supervisions inherent in the binocular stereo consistency between each pair of binocular images constructed with disparity-guided image warping. To this end, we additionally introduce a Gaussian opacity constraint which regularizes the Gaussian locations and avoids Gaussian redundancy for improving the robustness and efficiency of inferring 3D Gaussians from sparse views. Extensive experiments on the LLFF, DTU, and Blender datasets demonstrate that our method significantly outperforms the state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。