用语言增强3D高斯点云，实现大规模场景快速建模。
SLAG: Scalable Language-Augmented Gaussian Splatting
- 通过归一化加权平均直接生成3D高斯点的语言嵌入，无需损失函数。
- 16张显卡下嵌入计算速度比OpenGaussian快18倍，保持高质量。
- 适合资源受限机器人在搜救、智慧城市等场景中高效使用。
语言增强的场景表示在大规模机器人应用（如搜救、智慧城市、采矿）中具有巨大潜力。这些场景时间敏感且数据密集，要求快速编码大场景，同时需具备可扩展性。在计算资源有限的机器人上部署更增加挑战。为此，我们提出SLAG，一种用于语言增强高斯点云的多GPU框架，显著提升大场景嵌入的速度与可扩展性。方法通过SAM和CLIP将2D视觉-语言模型特征融入3D场景。不同于以往方法，SLAG无需损失函数计算每个高斯点的语言嵌入，而是基于3D高斯场景参数进行归一化加权平均，实现高度并行化编码。此外，引入向量数据库以高效存储和检索嵌入。实验表明，在16张GPU上，SLAG相比OpenGaussian在ScanNet和LERF数据集上嵌入计算速度提升18倍，同时保持嵌入质量。
原文摘要 · Abstract (English)
Language-augmented scene representations hold great promise for large-scale robotics applications such as search-and-rescue, smart cities, and mining. Many of these scenarios are time-sensitive, requiring rapid scene encoding while also being data-intensive, necessitating scalable solutions. Deploying these representations on robots with limited computational resources further adds to the challenge. To address this, we introduce SLAG, a multi-GPU framework for language-augmented Gaussian splatting that enhances the speed and scalability of embedding large scenes. Our method integrates 2D visual-language model features into 3D scenes using SAM and CLIP. Unlike prior approaches, SLAG eliminates the need for a loss function to compute per-Gaussian language embeddings. Instead, it derives embeddings from 3D Gaussian scene parameters via a normalized weighted average, enabling highly parallelized scene encoding. Additionally, we introduce a vector database for efficient embedding storage and retrieval. Our experiments show that SLAG achieves an 18 times speedup in embedding computation on a 16-GPU setup compared to OpenGaussian, while preserving embedding quality on the ScanNet and LERF datasets. For more details, visit our project website: https://slag-project.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。