arXiv:2505.15545cs.CV2025-05

用多视角投影生成伪标签,提升3D语义分割跨域性能

Multi-View Projection for Unsupervised Domain Adaptation in 3D Semantic Segmentation

  • 通过多视角渲染合成2D数据,生成目标域伪标签
  • 在nuScenes和SemanticKITTI上实现当前最优无监督域适应效果
  • 可仅用2D标注分割稀有类别,适合标注稀缺场景

3D语义分割在自动驾驶和道路基础设施分析中至关重要,但现有3D模型在不同数据集间部署时易受域偏移影响。本文提出一种无监督域适应方法:利用新提出的多视角投影框架生成伪标签,对目标域3D分割模型进行训练。该方法先将激光雷达扫描对齐为连贯3D场景,并从多个虚拟相机位姿渲染,生成大规模跨模态合成2D语义分割数据集。基于这些数据,在每种模态上训练一组2D分割模型(点云视图域)。推理时,每个场景处理大量视图,结果logits通过深度感知投票机制反投影至3D,生成最终逐点标签。这些标签用于微调目标域的3D分割模型。我们在nuScenes和SemanticKITTI上评估了Real-to-Real设置,在SynLidar上评估Simulation-to-Real设置。实验表明,本方法在真实域间无监督域适应任务中达到当前最优表现;同时展示了仅使用2D标注即可分割稀有类别(目标域无3D标注),利用源域3D标注辅助其他类别的应用能力。

原文摘要 · Abstract (English)

3D semantic segmentation plays a pivotal role in autonomous driving and road infrastructure analysis, yet state-of-the-art 3D models are prone to severe domain shift when deployed across different datasets. In this paper, we propose an Unsupervised Domain Adaptation approach where a 3D segmentation model is trained on the target dataset using pseudo-labels generated by a novel multi-view projection framework. Our approach first aligns Lidar scans into coherent 3D scenes and renders them from multiple virtual camera poses to create large-scale synthetic 2D semantic segmentation datasets in various modalities. The generated datasets are used to train an ensemble of 2D segmentation models in point cloud view domain on each modality. During inference, the models process a large amount of views per scene; the resulting logits are back-projected to 3D with a depth-aware voting scheme to generate final point-wise labels. These labels are then used to fine-tune a 3D segmentation model in the target domain. We evaluate our approach Real-to-Real on the nuScenes and SemanticKITTI datasets. We also evaluate it Simulation-to-Real with the SynLidar dataset. Our contributions are a novel method that achieves state-of-the-art results in Real-to-Real Unsupervised Domain Adaptation, and we also demonstrate an application of our method to segment rare classes, for which target 3D annotations are not available, by only using 2D annotations for those classes and leveraging 3D annotations for other classes in a source domain.

3D分割无监督学习域适应多视角

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。