arXiv:2510.14270cs.CVcs.GR2025-10

用2D模型增强3D高斯点云,提升细节还原与稀疏区域表现。

GauSSmart: Enhanced 3D Reconstruction through 2D Foundation Models and Geometric Filtering

  • 融合2D基础模型的语义特征与几何过滤,指导3D点云优化
  • 在三个数据集上均优于传统高斯点云方法,细节更清晰
  • 适合需要高保真3D重建的研究者与工业应用

场景重建是计算机视觉的核心挑战,神经辐射场(NeRF)和高斯点云(Gaussian Splatting)已取得显著进展。尽管高斯点云在大规模数据集上表现良好,但在覆盖稀疏区域仍难以捕捉精细细节或保持真实感,主要受限于稀疏的3D训练数据。本文提出GauSSmart,一种结合2D基础模型与3D高斯点云重建的混合方法。通过引入如DINO等2D视觉模型提供的语义特征监督、凸滤波等技术,利用2D分割先验与高维特征嵌入,引导高斯点云的稠密化与精细化,改善欠覆盖区域的表达并保留复杂结构细节。在三个数据集上的实验表明,GauSSmart在多数场景中持续优于现有高斯点云方法。结果验证了2D-3D融合策略的巨大潜力,表明将2D基础模型与3D重建流程有机结合,可有效克服单一方法的固有局限。

原文摘要 · Abstract (English)

Scene reconstruction has emerged as a central challenge in computer vision, with approaches such as Neural Radiance Fields (NeRF) and Gaussian Splatting achieving remarkable progress. While Gaussian Splatting demonstrates strong performance on large-scale datasets, it often struggles to capture fine details or maintain realism in regions with sparse coverage, largely due to the inherent limitations of sparse 3D training data. In this work, we propose GauSSmart, a hybrid method that effectively bridges 2D foundational models and 3D Gaussian Splatting reconstruction. Our approach integrates established 2D computer vision techniques, including convex filtering and semantic feature supervision from foundational models such as DINO, to enhance Gaussian-based scene reconstruction. By leveraging 2D segmentation priors and high-dimensional feature embeddings, our method guides the densification and refinement of Gaussian splats, improving coverage in underrepresented areas and preserving intricate structural details. We validate our approach across three datasets, where GauSSmart consistently outperforms existing Gaussian Splatting in the majority of evaluated scenes. Our results demonstrate the significant potential of hybrid 2D-3D approaches, highlighting how the thoughtful combination of 2D foundational models with 3D reconstruction pipelines can overcome the limitations inherent in either approach alone.

3D重建高斯点云2D基础模型语义引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。