用生成式先验修复视图稀疏下的3D场景重建缺陷
InceptionGS: Generative Bootstrapping for Large-Scale Gaussian Splatting under Unstructured View Sampling
- 通过自适应生成先验动态修复视图缺失区域
- 在真实大规模场景上实现高质量连续渲染
- 适合处理采集不完整的野外大场景数据
实现真正沉浸式的大型场景数字化需要在所有视角下保持一致且视觉愉悦的渲染效果。然而，由于场景复杂性、采集成本、遗漏或可达性限制，获取覆盖每个细节的多视角图像极为困难，导致采样视图高度无结构——多数区域覆盖良好，但某些区域不可避免地缺乏足够观测。现有基于重建的方法对视图稀缺敏感，而基于生成的方法则存在泛化性差、可控性弱和三维一致性不足的问题。为此，我们提出InceptionGS，通过巧妙平衡重建与生成来引导高斯点云的自举式优化。从初始高斯点云出发，InceptionGS合理重构并修复由视图稀疏引起的异常区域，同时保留其他区域质量，通过软性引入场景与视图自适应的生成先验。在真实世界大规模场景上的大量实验表明，该方法在处理无结构图像方面具有显著优势和广泛适用性，可有效提升高保真高斯点云渲染效果。
原文摘要 · Abstract (English)
Achieving truly immersive large-scale scene digitization necessitates consistent and visually pleasing rendering across all possible viewing perspectives. However, collecting multi-view images covering every fine detail of a large-scale scene is prohibitive due to scene complexity, capture cost, negligence, or accessibility constraints. As a result, the sampled views tend to be highly unstructured -- the majority of the scene is well covered yet certain regions inevitably lack sufficient observations. Existing reconstruction based methods are vulnerable to view scarcity while generation based approaches suffer from generalization, controllability, and 3D consistency issues. To address this challenge, we propose InceptionGS, which bootstraps Gaussian splatting by subtly balancing reconstruction and generation. Starting from an initial Gaussian splatting, InceptionGS reasonably rethinks and repairs problematic regions caused by view scarcity while preserving the quality elsewhere, by softly incorporating scene- and view-adaptive generative priors. Extensive experiments on real-world large-scale scenes demonstrate the superiority and broad applicability of our approach in handling unstructured imagery and boosting high-fidelity Gaussian splatting. Please refer to the supplementary video for better visual demonstrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。