arXiv:2509.23258cs.CV2025-09被引 4

用扩散模型生成完整场景,再用多视图验证真伪,解决稀疏视角重建的模糊问题。

OracleGS: Grounding Generative Priors for Sparse-View Gaussian Splatting

  • 先用3D扩散模型生成新视角,再用多视图立体匹配模型检验其几何可信度。
  • 在Mip-NeRF 360和NeRF Synthetic数据集上优于当前最优方法。
  • 适合需要高保真且无幻觉的稀疏视角3D重建研究者。

稀疏视角的新视角合成因严重几何模糊而本质上是病态问题。现有方法陷入权衡:回归模型几何准确但不完整,生成模型可补全场景却常引入结构不一致。我们提出OracleGS,一种新框架,使生成完整性与回归保真性兼得。不直接修补重建缺陷,而是采用“提出-验证”机制:首先利用预训练的3D感知扩散模型合成新视角,提出完整场景;随后将多视图立体(MVS)模型重用于3D感知的“判官”,通过注意力图揭示生成视角中由多视角证据支持的区域与因遮挡、纹理缺失或不一致导致高不确定性区域。该不确定性信号通过加权损失直接指导3D高斯点阵的优化。我们的方法将强大生成先验约束于多视角几何证据,过滤幻觉伪影,同时保留欠约束区域的合理补全,在Mip-NeRF 360和NeRF Synthetic等数据集上超越当前最优方法。

原文摘要 · Abstract (English)

Sparse-view novel view synthesis is fundamentally ill-posed due to severe geometric ambiguity. Current methods are caught in a trade-off: regressive models are geometrically faithful but incomplete, whereas generative models can complete scenes but often introduce structural inconsistencies. We propose OracleGS, a novel framework that reconciles generative completeness with regressive fidelity for sparse view Gaussian Splatting. Instead of using generative models to patch incomplete reconstructions, our "propose-and-validate" framework first leverages a pre-trained 3D-aware diffusion model to synthesize novel views to propose a complete scene. We then repurpose a multi-view stereo (MVS) model as a 3D-aware oracle to validate the 3D uncertainties of generated views, using its attention maps to reveal regions where the generated views are well-supported by multi-view evidence versus where they fall into regions of high uncertainty due to occlusion, lack of texture, or direct inconsistency. This uncertainty signal directly guides the optimization of a 3D Gaussian Splatting model via an uncertainty-weighted loss. Our approach conditions the powerful generative prior on multi-view geometric evidence, filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions, outperforming state-of-the-art methods on datasets including Mip-NeRF 360 and NeRF Synthetic.

3D重建扩散模型高斯点阵多视图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。