arXiv:2412.05908cs.CV2024-12被引 6

用4-6张图实现高保真3D重建,解决稀疏视角下的细节模糊问题

GBR: Generative Bundle Refinement for High-fidelity Gaussian Splatting with Enhanced Mesh Reconstruction

  • 融合神经束调整与生成式深度优化,提升稀疏视图下的几何精度
  • 在仅6视图条件下,对滕王阁、长城等真实场景实现细节丰富的重建
  • 适用于低资源场景重建,适合需要高质量3D建模的工业应用

高斯点阵因其高效的连续高斯原语表示与渲染能力,在三维场景建模中备受关注。然而,在稀疏视角输入下,由于几何与光度信息不足,其深度、形状和纹理存在歧义。本文提出GBR:基于生成式束调整的高保真高斯点阵与网格重建方法,仅需4-6张输入视图。GBR引入神经束调整模块,通过基础网络从无姿态图像生成初始3D点云与匹配点,再经束调整优化提升多视图一致性与点云精度;同时设计生成式深度细化模块,采用扩散策略增强几何细节并保持尺度一致。针对高斯点阵优化,提出多模态损失函数,包含深度与法向一致性、几何正则化及伪视图监督,提升稀疏条件下的鲁棒性。实验表明,GBR在多个主流数据集上显著优于现有方法。此外,仅用6视图即可对滕王阁、长城等大规模真实场景实现高细节重建与渲染。

原文摘要 · Abstract (English)

Gaussian splatting has gained attention for its efficient representation and rendering of 3D scenes using continuous Gaussian primitives. However, it struggles with sparse-view inputs due to limited geometric and photometric information, causing ambiguities in depth, shape, and texture. we propose GBR: Generative Bundle Refinement, a method for high-fidelity Gaussian splatting and meshing using only 4-6 input views. GBR integrates a neural bundle adjustment module to enhance geometry accuracy and a generative depth refinement module to improve geometry fidelity. More specifically, the neural bundle adjustment module integrates a foundation network to produce initial 3D point maps and point matches from unposed images, followed by bundle adjustment optimization to improve multiview consistency and point cloud accuracy. The generative depth refinement module employs a diffusion-based strategy to enhance geometric details and fidelity while preserving the scale. Finally, for Gaussian splatting optimization, we propose a multimodal loss function incorporating depth and normal consistency, geometric regularization, and pseudo-view supervision, providing robust guidance under sparse-view conditions. Experiments on widely used datasets show that GBR significantly outperforms existing methods under sparse-view inputs. Additionally, GBR demonstrates the ability to reconstruct and render large-scale real-world scenes, such as the Pavilion of Prince Teng and the Great Wall, with remarkable details using only 6 views.

3D重建高斯点阵稀疏视图生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。