无需训练,用超点图实现3D高斯点云的层级语义理解。
Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphs
- 直接从高斯原语构建超点图,划分空间紧凑且语义一致的区域。
- 相比现有方法,语义重建速度提升30倍以上,精度达顶尖水平。
- 适合需要快速、一致3D语义理解的视觉场景应用。
连接自然语言与3D几何是实现灵活、语言驱动场景理解的关键步骤。尽管3D高斯点云(3DGS)近年来实现了快速高质量的场景重建,但将开放词汇语义引入3DGS的研究仍面临挑战。现有方法通常需对每视图的2D语义特征图进行迭代优化,不仅效率低下,还导致多视角间语义不一致。为此,我们提出一种无需训练的框架,直接从高斯原语构建超点图。该图将场景划分为空间紧凑且语义连贯的区域,形成视角一致的3D实体,并为开放词汇理解提供结构化基础。基于图结构,设计高效重投影策略,将2D语义特征映射至超点,避免多视图迭代训练。最终表示具备强3D语义一致性,支持统一语义场内的粗粒度与细粒度开放词汇感知。大量实验表明,本方法在开放词汇分割任务上达到顶尖性能,语义场重建速度提升超过30倍。代码将开源于https://github.com/Atrovast/THGS。
原文摘要 · Abstract (English)
Bridging natural language and 3D geometry is a crucial step toward flexible, language-driven scene understanding. While recent advances in 3D Gaussian Splatting (3DGS) have enabled fast and high-quality scene reconstruction, research has also explored incorporating open-vocabulary understanding into 3DGS. However, most existing methods require iterative optimization over per-view 2D semantic feature maps, which not only results in inefficiencies but also leads to inconsistent 3D semantics across views. To address these limitations, we introduce a training-free framework that constructs a superpoint graph directly from Gaussian primitives. The superpoint graph partitions the scene into spatially compact and semantically coherent regions, forming view-consistent 3D entities and providing a structured foundation for open-vocabulary understanding. Based on the graph structure, we design an efficient reprojection strategy that lifts 2D semantic features onto the superpoints, avoiding costly multi-view iterative training. The resulting representation ensures strong 3D semantic coherence and naturally supports hierarchical understanding, enabling both coarse- and fine-grained open-vocabulary perception within a unified semantic field. Extensive experiments demonstrate that our method achieves state-of-the-art open-vocabulary segmentation performance, with semantic field reconstruction completed over $30\times$ faster. Our code will be available at https://github.com/Atrovast/THGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。