用分层高斯点云实现极稀疏无姿态360度场景重建
Free360: Layered Gaussian Splatting for Unbounded 360-Degree View Synthesis from Extremely Sparse and Unposed Views
- 分层高斯表示解决无界场景中稀疏视角的模糊问题
- 通过粗略几何恢复与层内优化，提升重建精度与完整性
- 适合做无相机参数约束下的全景三维重建的研究者
神经渲染在密集输入视角和精确位姿条件下已实现高质量三维重建与新视角合成。然而，在极稀疏、无姿态的无界360°场景中应用仍具挑战。本文提出一种新型神经渲染框架，实现无姿态、极稀疏视角下无界360°场景的三维重建。为解决稀疏输入下无界场景的空间模糊问题，我们采用基于分层高斯的表示方法，有效建模具有明确空间层级的场景。通过密集立体重建模型恢复粗略几何，并引入层特定的自举优化，以细化噪声并填补遮挡区域。此外，提出重建与生成的迭代融合机制及不确定性感知训练策略，促进两过程间的相互条件与增强。大量实验表明，本方法在渲染质量与表面重建精度上优于现有最先进方法。
原文摘要 · Abstract (English)
Neural rendering has demonstrated remarkable success in high-quality 3D neural reconstruction and novel view synthesis with dense input views and accurate poses. However, applying it to extremely sparse, unposed views in unbounded 360° scenes remains a challenging problem. In this paper, we propose a novel neural rendering framework to accomplish the unposed and extremely sparse-view 3D reconstruction in unbounded 360° scenes. To resolve the spatial ambiguity inherent in unbounded scenes with sparse input views, we propose a layered Gaussian-based representation to effectively model the scene with distinct spatial layers. By employing a dense stereo reconstruction model to recover coarse geometry, we introduce a layer-specific bootstrap optimization to refine the noise and fill occluded regions in the reconstruction. Furthermore, we propose an iterative fusion of reconstruction and generation alongside an uncertainty-aware training approach to facilitate mutual conditioning and enhancement between these two processes. Comprehensive experiments show that our approach outperforms existing state-of-the-art methods in terms of rendering quality and surface reconstruction accuracy. Project page: https://zju3dv.github.io/free360/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。