让3D场景理解支持任意语言查询，且内存更省。
GALA: Guided Attention with Language Alignment for Open Vocabulary Gaussian Splatting
- 用双可学习代码本设计跨注意力模块，实现语言与3D特征对齐。
- 在真实数据集上达成2D/3D开放词汇查询的优异性能。
- 无需为每个高斯点学特征，显著降低内存占用。
3D场景重建与理解日益受到关注，但现有方法仍难以从2D图像中捕捉细粒度、语言感知的3D表征。本文提出GALA，一种基于3D高斯喷溅（3DGS）的开放词汇3D场景理解新框架。GALA通过自监督对比学习蒸馏出场景特定的3D实例特征场。为拓展至广义语言特征场，引入核心贡献：一个包含两个可学习代码本的跨注意力模块，编码与视角无关的语义嵌入。该设计不仅保证实例内特征相似性，还支持无缝的2D与3D开放词汇查询。通过避免每个高斯点的高维特征学习，显著降低内存消耗。在真实世界数据集上的大量实验表明，GALA在2D与3D开放词汇任务上均表现出色。
原文摘要 · Abstract (English)
3D scene reconstruction and understanding have gained increasing popularity, yet existing methods still struggle to capture fine-grained, language-aware 3D representations from 2D images. In this paper, we present GALA, a novel framework for open-vocabulary 3D scene understanding with 3D Gaussian Splatting (3DGS). GALA distills a scene-specific 3D instance feature field via self-supervised contrastive learning. To extend to generalized language feature fields, we introduce the core contribution of GALA, a cross-attention module with two learnable codebooks that encode view-independent semantic embeddings. This design not only ensures intra-instance feature similarity but also supports seamless 2D and 3D open-vocabulary queries. It reduces memory consumption by avoiding per-Gaussian high-dimensional feature learning. Extensive experiments on real-world datasets demonstrate GALA's remarkable open-vocabulary performance on both 2D and 3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。