arXiv:2509.00379cs.CVcs.RO2025-09ICRA被引 1

用2D图像知识指导3D点云分割,免去昂贵标注

Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation

论文配图:Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation
图 1 · 摘自论文原文
  • 通过跨模态知识蒸馏,用2D图像模型指导3D点云分割
  • 在S3DIS和ScanNet上性能超越现有方法,提升达3.2%以上
  • 适合缺乏3D标注数据的自动驾驶场景应用

3D LiDAR点云语义分割在自动驾驶中至关重要。传统方法依赖大量人工标注的点云数据,成本高昂。相比之下,真实世界中的图像数据集具有丰富性和大规模优势。为减轻3D点云标注负担,本文提出两种跨模态知识蒸馏方法:无监督域适应知识蒸馏(UDAKD)与基于特征和语义的知识蒸馏(FSKD)。利用自动驾驶场景中相机与LiDAR的时空同步数据,直接将预训练的2D图像模型应用于未标注的2D图像数据。通过已知的2D-3D对应关系进行跨模态知识蒸馏,主动对齐3D网络输出与2D网络对应点的预测结果,从而无需3D标注。研究重点在于蒸馏过程中保留模态通用信息,过滤掉模态特异性细节。为此,采用自校准卷积作为域适应模块的基础。大量实验验证了所提方法的有效性,在S3DIS和ScanNet数据集上持续优于当前最优方法,最高提升达3.2%。

原文摘要 · Abstract (English)

Semantic segmentation of 3D LiDAR data plays a pivotal role in autonomous driving. Traditional approaches rely on extensive annotated data for point cloud analysis, incurring high costs and time investments. In contrast, realworld image datasets offer abundant availability and substantial scale. To mitigate the burden of annotating 3D LiDAR point clouds, we propose two crossmodal knowledge distillation methods: Unsupervised Domain Adaptation Knowledge Distillation (UDAKD) and Feature and Semantic-based Knowledge Distillation (FSKD). Leveraging readily available spatio-temporally synchronized data from cameras and LiDARs in autonomous driving scenarios, we directly apply a pretrained 2D image model to unlabeled 2D data. Through crossmodal knowledge distillation with known 2D-3D correspondence, we actively align the output of the 3D network with the corresponding points of the 2D network, thereby obviating the necessity for 3D annotations. Our focus is on preserving modality-general information while filtering out modality-specific details during crossmodal distillation. To achieve this, we deploy self-calibrated convolution on 3D point clouds as the foundation of our domain adaptation module. Rigorous experimentation validates the effectiveness of our proposed methods, consistently surpassing the performance of state-of-the-art approaches in the field.

3D分割知识蒸馏跨模态自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。