用图像分割模型生成点云伪标签,三重数据不足下仍保持高精度。
Data-Efficient Semantic Segmentation of 3D Point Clouds via Open-Vocabulary Image Segmentation-based Pseudo-Labeling
- 通过开放词汇图像分割模型直接从3D点云生成2D伪标签。
- 在仅几十个场景、每场景少于100个标注点下,性能超越现有方法。
- 两阶段过滤+类别平衡记忆库,有效缓解伪标签噪声与不平衡问题。
3D点云语义分割在诸多应用中至关重要。现实场景中,训练模型常面临三大数据瓶颈:训练场景稀缺、点级标注稀少,以及重建点云所依赖的2D图像序列缺失。现有高效算法通常仅解决其中一至两个问题,未兼顾三者。本文提出针对三重数据不足的训练框架——基于开放词汇图像分割的点云伪标签生成方法(PLOVIS)。PLOVIS利用开放词汇图像分割(OVIS)模型直接从训练点云生成2D伪标签,无需原始2D图像序列。为降低伪标签中的噪声与类别不平衡,引入两阶段过滤机制及类别平衡记忆库。第一阶段剔除低置信度伪标签,第二阶段移除可能错误标签,显著提升伪标签质量。在ScanNet、S3DIS、Toronto3D和Semantic3D四个基准数据集上,在真实数据稀缺条件下(仅数十个训练场景,每场景标注点数<100),PLOVIS持续优于标准微调与当前最优弱监督学习方法。代码将公开。
原文摘要 · Abstract (English)
Semantic segmentation of 3D point cloud scenes is a crucial task for various applications. In real-world scenarios, training segmentation models often faces three concurrent forms of data insufficiency: scarcity of training scenes, scarcity of point-level annotations, and absence of 2D image sequences from which point clouds were reconstructed. Existing data-efficient algorithms typically address only one or two of these challenges, leaving the joint treatment of all three unexplored. This paper proposes a data-efficient training framework specifically designed to address the three forms of data insufficiency. Our proposed algorithm, called Point pseudo-Labeling via Open-Vocabulary Image Segmentation (PLOVIS), leverages an Open-Vocabulary Image Segmentation (OVIS) model as a pseudo label generator to compensate for the lack of training data. PLOVIS creates 2D images for pseudo-labeling directly from training 3D point clouds, eliminating the need for 2D image sequences. To mitigate the inherent noise and class imbalance in pseudo labels, we introduce a two-stage filtering of pseudo labels combined with a class-balanced memory bank for effective training. The two-stage filtering mechanism first removes low-confidence pseudo labels, then discards likely incorrect pseudo labels, thereby enhancing the quality of pseudo labels. Experiments on four benchmark datasets, i.e., ScanNet, S3DIS, Toronto3D, and Semantic3D, under realistic data-scarce conditions (a few tens of training 3D scenes, each annotated with only <100 3D points) demonstrate that PLOVIS consistently outperforms existing methods including standard fine-tuning strategies and state-of-the-art weakly supervised learning algorithms. Code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。