用语言描述就能自动分割3D场景中的任意物体，无需人工标注。
OpenSplat3D: Open-Vocabulary 3D Instance Segmentation using Gaussian Splatting
- 将语义特征融入高斯点，实现细粒度3D实例分割。
- 结合SAM掩码与对比损失，精准区分不同物体实例。
- 支持自然语言驱动识别，适合开放词汇场景理解。
3D高斯喷溅（3DGS）已成为神经场景重建的强大表示方法，能在保持计算效率的同时生成高质量的新视角图像。本文提出OpenSplat3D，首次在无需人工标注的前提下，实现基于开放词汇的3D实例分割。该方法通过特征喷溅技术将语义信息关联至每个高斯点，实现细粒度场景理解。我们引入段落任何模型（SAM）的实例掩码，并采用对比损失约束实例特征，提升分割精度。同时融合视觉-语言模型的语言嵌入，实现以自然语言为驱动的任意物体识别。在LERF-mask、LERF-OVS及完整ScanNet++验证集上均取得有效结果，证明了方法的优越性。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has emerged as a powerful representation for neural scene reconstruction, offering high-quality novel view synthesis while maintaining computational efficiency. In this paper, we extend the capabilities of 3DGS beyond pure scene representation by introducing an approach for open-vocabulary 3D instance segmentation without requiring manual labeling, termed OpenSplat3D. Our method leverages feature-splatting techniques to associate semantic information with individual Gaussians, enabling fine-grained scene understanding. We incorporate Segment Anything Model instance masks with a contrastive loss formulation as guidance for the instance features to achieve accurate instance-level segmentation. Furthermore, we utilize language embeddings of a vision-language model, allowing for flexible, text-driven instance identification. This combination enables our system to identify and segment arbitrary objects in 3D scenes based on natural language descriptions. We show results on LERF-mask and LERF-OVS as well as the full ScanNet++ validation set, demonstrating the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。