arXiv:2509.05144cs.CV2025-09AAAI被引 3

通过分割与生长提升3D实例分割精度,解决2D到3D转换中的误差累积问题。

SGS-3D: High-Fidelity 3D Instance Segmentation via Reliable Semantic Mask Splitting and Growing

  • 先用几何原语净化并拆分模糊的2D升维掩码,再基于空间连续性生长成完整实例。
  • 在ScanNet200、ScanNet++和KITTI-360上显著提升分割准确率与鲁棒性。
  • 无需训练,可适配不同场景,尤其擅长处理语义模糊的物体边界。

精确的3D实例分割对高质量场景理解至关重要。然而,基于2D-to-3D升维的方法因语义引导模糊和深度约束不足,导致升维过程产生累积误差,难以实现精细实例级分割。为此,我们提出一种名为SGS-3D的新框架,采用“先分割后生长”的策略:首先利用几何原语净化并拆分模糊的升维掩码,再在场景中生长为完整实例。不同于依赖原始升维掩码而牺牲精度的现有方法,SGS-3D是一种无需训练的优化方法,联合融合语义与几何信息,实现两层表征的有效协同。具体而言,在语义引导方面,引入基于3D几何原语共现性的掩码过滤策略,识别并剔除模糊掩码,增强与3D物体实例的语义一致性;在几何精修方面,结合空间连续性与高层特征,尤其在区分不同物体的语义模糊情况下构建细粒度实例。在ScanNet200、ScanNet++和KITTI-360上的实验表明,SGS-3D显著提升分割精度与鲁棒性,对预训练模型产生的不准确掩码具有较强容忍度,生成高保真物体实例,并在多种室内外环境下保持强泛化能力。代码已开源。

原文摘要 · Abstract (English)

Accurate 3D instance segmentation is crucial for high-quality scene understanding in the 3D vision domain. However, 3D instance segmentation based on 2D-to-3D lifting approaches struggle to produce precise instance-level segmentation, due to accumulated errors introduced during the lifting process from ambiguous semantic guidance and insufficient depth constraints. To tackle these challenges, we propose splitting and growing reliable semantic mask for high-fidelity 3D instance segmentation (SGS-3D), a novel "split-then-grow" framework that first purifies and splits ambiguous lifted masks using geometric primitives, and then grows them into complete instances within the scene. Unlike existing approaches that directly rely on raw lifted masks and sacrifice segmentation accuracy, SGS-3D serves as a training-free refinement method that jointly fuses semantic and geometric information, enabling effective cooperation between the two levels of representation. Specifically, for semantic guidance, we introduce a mask filtering strategy that leverages the co-occurrence of 3D geometry primitives to identify and remove ambiguous masks, thereby ensuring more reliable semantic consistency with the 3D object instances. For the geometric refinement, we construct fine-grained object instances by exploiting both spatial continuity and high-level features, particularly in the case of semantic ambiguity between distinct objects. Experimental results on ScanNet200, ScanNet++, and KITTI-360 demonstrate that SGS-3D substantially improves segmentation accuracy and robustness against inaccurate masks from pre-trained models, yielding high-fidelity object instances while maintaining strong generalization across diverse indoor and outdoor environments. Code is available at https://github.com/wangchaolei7/SGS-3D.

3D分割实例分割几何融合无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。