arXiv:2605.08925cs.CV2026-05

仅需一次点击即可精准分割3D点云中的多个物体。

ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddings

论文配图:ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddings
图 1 · 摘自论文原文
  • 基于点变换器与可学习语义嵌入,单次前向传播处理多物体点击。
  • 相比强基线提升20%以上mIoU,跨数据集评估增益达8-10%。
  • 适用于机器人操作、导航等实时场景,交互效率高。

交互式分割通过用户点击逐步优化预测,能高效生成标签,尤其在全监督标签成本高或需泛化到未见类别时至关重要。现有3D交互方法受限:多数按顺序处理,每轮仅预测一个物体且输出二值掩码;部分近期方法依赖2D基础模型与相机对齐以弥合2D-3D鸿沟。为此,我们提出一种新框架,直接在稀疏随机下采样的3D点上运行,并在单次前向传播中处理多个物体的点击。该框架由点变换器编码器与分层掩码解码器组成,融合多层级裁剪合并操作,条件依赖可学习语义嵌入。不同于以往需每次手动修正后重推模型的方法,本方法联合推理所有点击查询,建模实例间关系,通过空间与语义嵌入同时优化空间掩码与语义预测。大量实验表明,相较强基线,模型在mIoU上提升超20%,在跨数据集评估中一点击/实例设置下获得8-10%增益,常仅需单次点击即完成分割。该方法为交互式3D实例分割提供通用且高效的解决方案,尤其适合机器人操作、导航及快速3D语义标注等实时应用。

原文摘要 · Abstract (English)

Interactive segmentation allows efficient label generation by leveraging user-provided clicks to progressively refine predictions, which is critical when fully supervised labels are costly or generalization to unseen classes is needed. Existing 3D interactive methods are limited: most operate sequentially, predicting only one object per iteration with binary masks, while several recent approaches depend on 2D foundation models and camera alignment to bridge the 2D-3D gap. To address these limitations, we propose a novel interactive segmentation framework that operates directly on sparse, randomly downsampled 3D points and processes multiple object clicks in a single forward pass. Our framework consists of a point Transformer-based encoder and a hierarchical mask decoder, which integrates multi-level crop-and-merge operations conditioned on learnable semantic embeddings. Unlike prior interactive approaches that require repeated model updates after each manually corrective click, our method jointly reasons over all click queries, modeling inter-instance relationships and refining both spatial masks and semantic predictions through spatial and semantic embeddings. Extensive experiments demonstrate that our model improves the mIoU metric by over 20 percent compared to strong baselines and achieves 8-10 percent gains under cross-dataset evaluation for a one-click per instance setting, often requiring only a single click per object. Our approach provides a generalizable and efficient solution for interactive 3D instance segmentation, particularly suitable for real-time applications such as robotic manipulation, navigation, and rapid 3D semantic annotation.

3D分割交互式点云

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。