arXiv:2508.04962cs.CVcs.GR2025-08

让人类少量标注辅助点云分割,轻松识别新旧类别。

Open-world Point Cloud Semantic Segmentation: A Human-in-the-loop Framework

论文配图:Open-world Point Cloud Semantic Segmentation: A Human-in-the-loop Framework
图 1 · 摘自论文原文
  • 直接在查询数据构建类别原型,避免样本分布偏移问题。
  • 仅需少量点击标注(如每类1次),就能达到顶尖分割效果。
  • 适合需要快速适应新类别的实际场景,如自动驾驶与机器人导航。

开放世界点云语义分割(OW-Seg)旨在真实场景中对基础类和新类别点进行标签预测。现有方法依赖资源密集的离线增量学习或密集标注的支持数据,实用性受限。为此,我们提出HOW-Seg,首个面向OW-Seg的人机协同框架。具体地,我们直接在查询数据上构建类别原型,避免因支持集与查询集间类内分布差异导致的原型偏差。通过稀疏人类标注引导,HOW-Seg实现对基础类与新类的基于原型的分割。针对初始原型粒度不足的问题,引入分层原型消歧机制,细化模糊原型(对应不同类别标注)。为进一步增强上下文感知,采用密集条件随机场(CRF)优化精炼后的原型标签分配。通过迭代人类反馈,HOW-Seg动态提升预测质量,实现对基础与新类别的高质量分割。实验表明,在稀疏标注(如每类1次点击)条件下,HOW-Seg性能匹配或超越最先进的广义少样本分割(GFS-Seg)方法(5-shot设置)。使用先进主干网络(如Stratified Transformer)及更密集标注(每子场景10次点击)时,HOW-Seg在S3DIS上达85.27% mIoU,ScanNetv2上达66.37% mIoU,显著优于现有方法。

原文摘要 · Abstract (English)

Open-world point cloud semantic segmentation (OW-Seg) aims to predict point labels of both base and novel classes in real-world scenarios. However, existing methods rely on resource-intensive offline incremental learning or densely annotated support data, limiting their practicality. To address these limitations, we propose HOW-Seg, the first human-in-the-loop framework for OW-Seg. Specifically, we construct class prototypes, the fundamental segmentation units, directly on the query data, avoiding the prototype bias caused by intra-class distribution shifts between the support and query data. By leveraging sparse human annotations as guidance, HOW-Seg enables prototype-based segmentation for both base and novel classes. Considering the lack of granularity of initial prototypes, we introduce a hierarchical prototype disambiguation mechanism to refine ambiguous prototypes, which correspond to annotations of different classes. To further enrich contextual awareness, we employ a dense conditional random field (CRF) upon the refined prototypes to optimize their label assignments. Through iterative human feedback, HOW-Seg dynamically improves its predictions, achieving high-quality segmentation for both base and novel classes. Experiments demonstrate that with sparse annotations (e.g., one-novel-class-one-click), HOW-Seg matches or surpasses the state-of-the-art generalized few-shot segmentation (GFS-Seg) method under the 5-shot setting. When using advanced backbones (e.g., Stratified Transformer) and denser annotations (e.g., 10 clicks per sub-scene), HOW-Seg achieves 85.27% mIoU on S3DIS and 66.37% mIoU on ScanNetv2, significantly outperforming alternatives.

点云分割人机协同开放世界少样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。