五秒内完成高效3D场景理解，支持自然语言查询新物体。
LightSplat: Fast and Memory-Efficient Open-Vocabulary 3D Scene Understanding in Five Seconds
- 用2字节语义索引替代密集特征，直接注入多视角图像生成3D表示
- 单步聚类实现几何与语义关联，推理速度提升50-400倍，内存降低64倍
- 无需训练，适合实时交互式3D应用和资源受限设备
开放词汇3D场景理解使用户能通过自然语言分割复杂3D环境中的新物体。但现有方法因迭代优化和稠密的每个高斯特征分配，仍存在速度慢、内存占用高、结构复杂的问题。为此，我们提出LightSplat——一种无需训练的快速、轻量级框架，从多视图图像中将紧凑的2字节语义索引注入3D表示。仅对显著区域分配语义索引，并通过轻量级索引-特征映射管理，彻底消除昂贵的特征优化与存储开销。进一步通过单步聚类确保语义一致性，高效链接3D中几何与语义相关的掩码。我们在LERF-OVS、ScanNet和DL3DV-OVS上的复杂室内外场景进行评估，结果表明LightSplat达到当前最佳性能，速度提升最高达50-400倍，内存降低64倍，实现可扩展的语言驱动3D理解。
原文摘要 · Abstract (English)
Open-vocabulary 3D scene understanding enables users to segment novel objects in complex 3D environments through natural language. However, existing approaches remain slow, memory-intensive, and overly complex due to iterative optimization and dense per-Gaussian feature assignments. To address this, we propose LightSplat, a fast and memory-efficient training-free framework that injects compact 2-byte semantic indices into 3D representations from multi-view images. By assigning semantic indices only to salient regions and managing them with a lightweight index-feature mapping, LightSplat eliminates costly feature optimization and storage overhead. We further ensure semantic consistency and efficient inference via single-step clustering that links geometrically and semantically related masks in 3D. We evaluate our method on LERF-OVS, ScanNet, and DL3DV-OVS across complex indoor-outdoor scenes. As a result, LightSplat achieves state-of-the-art performance with up to 50-400x speedup and 64x lower memory, enabling scalable language-driven 3D understanding. For more details, visit our project page https://vision3d-lab.github.io/lightsplat/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。