arXiv:2501.09485cs.CV2025-01中稿 · ACCV2024

修复点云坐标与量化设计,让图像转点云学习性能提升16%以上。

The Devil is in the Details: Simple Remedies for Image-to-LiDAR Representation Learning

  • 改进点云坐标系和量化方式,减少空间误差
  • 在nuScenes上3D语义分割提升16%,KITTI上检测提升13%
  • 适合做自动驾驶感知模型的工程师和研究者

LiDAR是自动驾驶中关键传感器,常与摄像头协同使用。借助摄像头-点云配置及图像表征学习进展,已有研究展示了图像到点云知识迁移的潜力。这些工作主要聚焦于损失函数设计,以将预训练2D图像表征迁移到3D模型中。然而,其他设计要素却鲜受关注。本文发现,点云坐标系、输入接口下的量化方式以及数据利用策略等基础设计远比损失函数重要,却被以往工作忽视。我们针对空间与时间轴上的被忽略设计进行分析并提出简单解决方案:空间上,现有方法采用圆柱坐标与体素大小,未考虑其与常用稀疏卷积输入接口的副作用,导致3D模型中出现空间量化误差;时间上,为避免繁琐数据对齐,现有方法舍弃不同步数据,仅使用少量时序同步数据,限制了可用性。我们提出相应修正后,在nuScenes数据集上3D语义分割性能提升16%,KITTI数据集上3D目标检测性能提升13%。

原文摘要 · Abstract (English)

LiDAR is a crucial sensor in autonomous driving, commonly used alongside cameras. By exploiting this camera-LiDAR setup and recent advances in image representation learning, prior studies have shown the promising potential of image-to-LiDAR distillation. These prior arts focus on the designs of their own losses to effectively distill the pre-trained 2D image representations into a 3D model. However, the other parts of the designs have been surprisingly unexplored. We find that fundamental design elements, e.g., the LiDAR coordinate system, quantization according to the existing input interface, and data utilization, are more critical than developing loss functions, which have been overlooked in prior works. In this work, we show that simple fixes to these designs notably outperform existing methods by 16% in 3D semantic segmentation on the nuScenes dataset and 13% in 3D object detection on the KITTI dataset in downstream task performance. We focus on overlooked design choices along the spatial and temporal axes. Spatially, prior work has used cylindrical coordinate and voxel sizes without considering their side effects yielded with a commonly deployed sparse convolution layer input interface, leading to spatial quantization errors in 3D models. Temporally, existing work has avoided cumbersome data curation by discarding unsynced data, limiting the use to only the small portion of data that is temporally synced across sensors. We analyze these effects and propose simple solutions for each overlooked aspect.

点云学习自动驾驶表征迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。