用图像与点云对齐生成高质量伪标签,提升无精细标注的室内场景分割效果。
High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotation

- 通过2D-3D特征对齐,融合图像信息增强点云语义学习。
- 引入区域-点语义一致性模块,提升伪标签准确性。
- 适用于缺乏点级标注的室内点云分割任务,适合弱监督学习研究者。
本文研究在场景级标注下的室内点云语义分割,该问题相较于依赖稀疏点级标签的方法仍较少被探索。由于缺乏精确的点级标签,现有方法先生成点级伪标签,再用于训练分割模型。然而,仅基于场景级标注生成每个点的准确伪标签面临巨大挑战,严重影响分割性能。为此,本文提出一种高质量伪标签生成框架,通过挖掘多模态信息与区域-点语义一致性来提升精度。具体地,采用跨模态特征引导模块,利用2D-3D对应关系对齐点云特征与对应2D图像像素,辅助点云特征学习;为缓解场景级标注带来的困难,引入区域-点语义一致性模块,通过点级语义的区域投票策略生成区域语义,并用于指导点级语义预测。借助上述模块,本方法可在训练中修正错误的点级语义预测,获得高质量伪标签。在ScanNet v2和S3DIS数据集上的大量实验表明,该方法显著优于先前工作。全面的消融实验验证了各组件的有效性。代码已开源:https://github.com/LHDuan/WSegPC。
原文摘要 · Abstract (English)
This paper investigates indoor point cloud semantic segmentation under scene-level annotation, which is less explored compared to methods relying on sparse point-level labels. In the absence of precise point-level labels, current methods first generate point-level pseudo-labels, which are then used to train segmentation models. However, generating accurate pseudo-labels for each point solely based on scene-level annotations poses a considerable challenge, substantially affecting segmentation performance. Consequently, to enhance accuracy, this paper proposes a high-quality pseudo-label generation framework by exploring contemporary multi-modal information and region-point semantic consistency. Specifically, with a cross-modal feature guidance module, our method utilizes 2D-3D correspondences to align point cloud features with corresponding 2D image pixels, thereby assisting point cloud feature learning. To further alleviate the challenge presented by the scene-level annotation, we introduce a region-point semantic consistency module. It produces regional semantics through a region-voting strategy derived from point-level semantics, which are subsequently employed to guide the point-level semantic predictions. Leveraging the aforementioned modules, our method can rectify inaccurate point-level semantic predictions during training and obtain high-quality pseudo-labels. Significant improvements over previous works on ScanNet v2 and S3DIS datasets under scene-level annotation can demonstrate the effectiveness. Additionally, comprehensive ablation studies validate the contributions of our approach's individual components. The code is available at https://github.com/LHDuan/WSegPC .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。