arXiv:2506.06822cs.CVcs.AI2025-06被引 3

让3D语言场景理解更连贯,支持多层级语义查询。

Hi-LSplat: Hierarchical 3D Language Gaussian Splatting

  • 构建分层实例聚类的3D语义树,统一视图间语义表达。
  • 在两个新构建的分层语义数据集上实现领先分割与定位性能。
  • 适合需要精细语义理解的3D场景分析任务,如智能导航、虚拟交互。

基于高斯点云的3D语言场建模近年受到关注,但现有方法依赖视图相关的2D基础模型优化3D语义,缺乏统一的3D表示,导致视图不一致。同时,开放词汇挑战使物体与关系描述存在矛盾,阻碍层次化语义理解。本文提出Hi-LSplat,一种视图一致的分层语言高斯点云方法,用于3D开放词汇查询。通过构建分层实例聚类的3D层次语义树,将2D特征提升至3D,解决2D语义特征引发的视图不一致性问题。引入实例级与部件级对比损失,捕捉全方位的层次语义表征。特别地,构建了两个分层语义数据集以更准确评估模型对不同语义层级的区分能力。大量实验表明,该方法在3D开放词汇分割与定位任务中表现优越,在分层语义数据集上的优异表现证明其具备捕捉复杂3D场景层次语义的能力。

原文摘要 · Abstract (English)

Modeling 3D language fields with Gaussian Splatting for open-ended language queries has recently garnered increasing attention. However, recent 3DGS-based models leverage view-dependent 2D foundation models to refine 3D semantics but lack a unified 3D representation, leading to view inconsistencies. Additionally, inherent open-vocabulary challenges cause inconsistencies in object and relational descriptions, impeding hierarchical semantic understanding. In this paper, we propose Hi-LSplat, a view-consistent Hierarchical Language Gaussian Splatting work for 3D open-vocabulary querying. To achieve view-consistent 3D hierarchical semantics, we first lift 2D features to 3D features by constructing a 3D hierarchical semantic tree with layered instance clustering, which addresses the view inconsistency issue caused by 2D semantic features. Besides, we introduce instance-wise and part-wise contrastive losses to capture all-sided hierarchical semantic representations. Notably, we construct two hierarchical semantic datasets to better assess the model's ability to distinguish different semantic levels. Extensive experiments highlight our method's superiority in 3D open-vocabulary segmentation and localization. Its strong performance on hierarchical semantic datasets underscores its ability to capture complex hierarchical semantics within 3D scenes.

3D生成语言理解高斯溅射分层语义

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。