用高斯方法快速实现3D实例分割，无需训练即可通用
Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation
- 直接从2D分割图融合特征到3D高斯场，无需每场景重训
- 速度比现有方法快一个数量级，支持任意已有3DGS重建
- 适合需要快速提取3D资产或做新视角合成的研究者
我们提出Lifting By Gaussians（LBG），一种针对3D高斯溅射辐射场（3DGS）的开放世界3D实例分割新方法。3DGS近年来成为高质量新视角合成的高效显式替代方案。本方法直接将来自SAM（或FastSAM等）的2D分割掩码，以及来自CLIP和DINOv2的特征，融合至3DGS（或类似高斯辐射场如2DGS）。与以往方法不同，LBG无需每场景训练，可无缝应用于任意现有3DGS重建。该方法不仅比现有方法快一个数量级且更简单，还高度模块化，无需特定高斯参数化即可实现3D语义分割。此外，其在2D语义新视角合成与3D资产提取上均取得更优结果，同时保持灵活性与效率。我们还提出一种新评估方法，用于独立评价3D辐射场分割中生成的单个3D资产。
原文摘要 · Abstract (English)
We introduce Lifting By Gaussians (LBG), a novel approach for open-world instance segmentation of 3D Gaussian Splatted Radiance Fields (3DGS). Recently, 3DGS Fields have emerged as a highly efficient and explicit alternative to Neural Field-based methods for high-quality Novel View Synthesis. Our 3D instance segmentation method directly lifts 2D segmentation masks from SAM (alternately FastSAM, etc.), together with features from CLIP and DINOv2, directly fusing them onto 3DGS (or similar Gaussian radiance fields such as 2DGS). Unlike previous approaches, LBG requires no per-scene training, allowing it to operate seamlessly on any existing 3DGS reconstruction. Our approach is not only an order of magnitude faster and simpler than existing approaches; it is also highly modular, enabling 3D semantic segmentation of existing 3DGS fields without requiring a specific parametrization of the 3D Gaussians. Furthermore, our technique achieves superior semantic segmentation for 2D semantic novel view synthesis and 3D asset extraction results while maintaining flexibility and efficiency. We further introduce a novel approach to evaluate individually segmented 3D assets from 3D radiance field segmentation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。