arXiv:2510.12099cs.CV2025-10被引 5

用几何引导生成先验,让3D场景重建更准更一致。

G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior

  • 利用平面结构估计精确深度图,提供可靠几何监督。
  • 在未观测区域重建误差降低27%,多视角一致性显著提升。
  • 支持单视角输入,适用于室内外真实场景重建。

尽管预训练扩散模型的生成先验已被用于3D场景重建,现有方法仍面临两大挑战:一是缺乏可靠的几何监督,导致即使在已观测区域也难以生成高质量重建;二是缺乏有效机制缓解多视角图像不一致,引发严重形状-外观歧义,损害场景几何。本文提出,准确几何是有效利用生成模型提升3D重建的基础。我们首先利用平面结构普遍性推导出度量尺度的深度图,为观测与未观测区域提供可靠监督。进一步将该几何引导融入生成全流程,改进可见性掩码估计、指导新视角选择,并在视频扩散模型修复时增强多视角一致性,实现高精度且一致的场景补全。在Replica、ScanNet++、DeepBlending和Mip-NeRF 360上的实验表明,本方法在几何与外观重建上均优于现有基线,尤其在未观测区域表现突出。此外,方法自然支持单视角输入和无姿态视频,在室内外场景中具备强泛化能力,具有实际应用价值。

原文摘要 · Abstract (English)

Despite recent advances in leveraging generative prior from pre-trained diffusion models for 3D scene reconstruction, existing methods still face two critical limitations. First, due to the lack of reliable geometric supervision, they struggle to produce high-quality reconstructions even in observed regions, let alone in unobserved areas. Second, they lack effective mechanisms to mitigate multi-view inconsistencies in the generated images, leading to severe shape-appearance ambiguities and degraded scene geometry. In this paper, we identify accurate geometry as the fundamental prerequisite for effectively exploiting generative models to enhance 3D scene reconstruction. We first propose to leverage the prevalence of planar structures to derive accurate metric-scale depth maps, providing reliable supervision in both observed and unobserved regions. Furthermore, we incorporate this geometry guidance throughout the generative pipeline to improve visibility mask estimation, guide novel view selection, and enhance multi-view consistency when inpainting with video diffusion models, resulting in accurate and consistent scene completion. Extensive experiments on Replica, ScanNet++, DeepBlending and Mip-NeRF 360 show that our method consistently outperforms existing baselines in both geometry and appearance reconstruction, particularly for unobserved regions. Moreover, our method naturally supports single-view inputs and unposed videos, with strong generalizability in both indoor and outdoor scenarios with practical real-world applicability. The project page is available at https://dali-jack.github.io/g4splat-web/.

3D重建生成先验几何引导扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。