解决3D高斯点云几何失真问题,提升视角合成与重建精度。
G3Splat: Geometrically Consistent Generalizable Gaussian Splatting
- 引入可微分几何先验,强制高斯点沿视线方向且贴合表面朝向。
- 在RE10K上训练后,新视角深度和网格重建性能超越现有方法。
- 适配多种主干网络,适用于全秩3DGS与类面片2DGS表示。
3D高斯已成为实时渲染与高质量新视角合成的强大场景表示。这推动了泛化性点云生成方法的发展——即通过前馈几何预测网络,从一组图像生成每个像素的3D高斯。然而,大多数泛化性点云管道主要依赖视图合成损失来预测高斯的位置、方向、各向异性尺度、不透明度及外观。我们发现这一学习目标约束不足:仅用视图合成训练的模型生成的高斯方向与尺度缺乏几何意义,导致尽管视图合成表现尚可,但其高斯分布普遍存在几何不准与错位。为此,我们提出G3Splat,一种几何一致的泛化性高斯点云框架,通过可微分几何先验对预测的3D高斯施加约束,使学习问题合理化。这些先验确保每个像素的高斯保持在其观测射线上,并与局部曲面一致对齐。该方法架构无关,可融入任意已有几何主干,以及不同场景表示。我们在DUSt3R与VGGT两种主干上测试,实现像素对齐的全秩3DGS与类面片2DGS。在RE10K上训练后,G3Splat在ACID与ScanNet等数据集上的新视角深度、网格重建与相对位姿估计均达到当前最优,同时保持优异的视图合成质量。代码与预训练模型已开源。
原文摘要 · Abstract (English)
3D Gaussians have become a powerful scene representation for real-time splatting and high-quality novel-view synthesis. This has motivated generalizable splatting -- methods that adapt feed-forward geometry prediction networks to produce per-pixel Gaussians from a set of images. However, most generalizable splatting pipelines are supervised primarily through a view-synthesis loss to predict Gaussian orientation, anisotropic scale, opacity, and appearance in addition to their locations. We show that this learning objective is under-constrained. Models trained with view synthesis alone produce splats whose orientations and scales have no geometric connotation. The result is that, while producing decent view-synthesis performance, nearly all generalizable splatting methods produce geometrically inaccurate and misaligned Gaussians. We introduce G3Splat, a geometry-consistent generalizable splatting framework that addresses these degeneracies through differentiable geometric priors on the predicted 3D Gaussians, making the learning problem well-posed. These priors encourage the per-pixel splats to remain on their viewing rays and to orient themselves in accordance with local surfaces. Our priors are architecture-agnostic and can be incorporated into any previously studied geometric backbone for generalizable splatting, as well as different scene representations. We test G3Splat with both DUSt3R-style and VGGT-style backbones to predict pixel-aligned full-rank 3DGS as well as surfel-like 2DGS. Trained on RE10K, G3Splat produces Gaussian splats with significantly higher geometric fidelity than baselines, providing state-of-the-art novel-view depth, mesh reconstruction, and relative pose estimation performance while preserving novel-view synthesis quality, as evaluated on datasets such as ACID and ScanNet. Code and pretrained models are released on our project page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。