让3D高斯地图会说话，实时支持语言理解与智能压缩。
LEGO-SLAM: Language-Embedded Gaussian Optimization SLAM
- 用自适应编码器将语言特征压缩到16维，降低内存与渲染开销。
- 通过语义冗余检测，地图点数减少58%仍保持画质。
- 利用已有语言特征做回环检测，无需额外模型，适合机器人交互。
3D高斯点阵（3DGS）的进展使SLAM系统能构建逼真的三维地图，但这类地图缺乏机器人交互所需的开放词汇语义理解。将语言特征融入SLAM仍面临挑战：高维特征存储耗内存、渲染慢，而现有静态方法难以适应新环境。本文提出LEGO-SLAM，一种基于3DGS的实时开放词汇SLAM框架。核心是一个场景自适应的自动编码器，可将高维语言嵌入压缩至16维特征空间，显著降低每个高斯点的内存占用并加速渲染。该编码器在线适应未知场景，实现动态泛化。压缩后的特征还支持语言引导的剪枝策略，识别语义冗余，使高斯点数量最多减少58%，同时保持渲染质量。此外，我们提出基于语言的回环检测方法，复用映射过程中已提取的语言特征，无需额外检测模型。实验表明，LEGO-SLAM在15帧/秒下达到与主流方法相当的建图质量与跟踪精度，兼具开放词汇能力。
原文摘要 · Abstract (English)
Recent advances in 3D Gaussian Splatting (3DGS) have enabled Simultaneous Localization and Mapping (SLAM) systems to build photorealistic maps. However, these maps lack the open-vocabulary semantic understanding required for robotic interaction. Integrating language features into SLAM remains a significant challenge, as storing high-dimensional features incurs excessive memory and rendering overhead, while existing methods with static models lack adaptability for novel environments. We propose LEGO-SLAM (Language-Embedded Gaussian Optimization SLAM), a framework that achieves real-time, open-vocabulary mapping within a 3DGS-based SLAM system. At the core of our method is a scene-adaptive autoencoder that distills high-dimensional language embeddings into a compact 16-dimensional feature space, reducing the memory per Gaussian and accelerating rendering. Unlike static approaches, our encoder adapts online to unseen scenes. These compact features also enable a language-guided pruning strategy that identifies semantic redundancy, reducing the map's Gaussian count by up to 58% while maintaining rendering quality. Furthermore, we introduce a language-based loop detection approach that reuses the language features already extracted for mapping, eliminating the need for a separate detection model. Experiments demonstrate that LEGO-SLAM achieves competitive mapping quality and tracking accuracy, all while providing open-vocabulary capabilities at 15 FPS. Our project page is available at https://lab-of-ai-and-robotics.github.io/LEGO-SLAM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。