arXiv:2506.02914cs.CV2025-06

用专家指南自动标注3D激光雷达数据,提升检测精度。

Auto-Annotation with Expert-Crafted Guidelines: A Study through 3D LiDAR Detection Benchmark

  • 基于2D图像和文本指南,用基础模型实现3D目标检测
  • 在nuScenes基准上将mAP从12.1提升至25.4
  • 适合自动驾驶数据标注自动化研究者

数据标注对机器学习至关重要。当前依赖普通人工按专家制定的指南标注数据,过程耗时且昂贵。为此,我们探索基于专家指南的自动标注。构建了AutoExpert基准,基于nuScenes数据集,包含18类物体的细粒度语言描述与少量视觉示例,要求在激光雷达数据中以3D框标注目标。但指南未提供激光雷达示例,导致方法需从少样本的图像和文本中学习,完成激光雷达3D检测。挑战在于数据模态与标注任务差异。我们采用简单管道:先用基础模型在RGB图像中进行2D检测与分割,再利用已知传感器位姿将2D结果投影到3D空间,最终生成3D框。通过逐步优化各环节,最终使3D检测mAP达到25.4,显著优于此前12.1的水平。

原文摘要 · Abstract (English)

Data annotation is crucial for developing machine learning solutions. The current paradigm is to hire ordinary human annotators to annotate data instructed by expert-crafted guidelines. As this paradigm is laborious, tedious, and costly, we are motivated to explore auto-annotation with expert-crafted guidelines. To this end, we first develop a supporting benchmark AutoExpert by repurposing the established nuScenes dataset, which has been widely used in autonomous driving research and provides authentic expert-crafted guidelines. The guidelines define 18 object classes using both nuanced language descriptions and a few visual examples, and require annotating objects in LiDAR data with 3D cuboids. Notably, the guidelines do not provide LiDAR visuals to demonstrate how to annotate. Therefore, AutoExpert requires methods to learn on few-shot labeled images and texts to perform 3D detection in LiDAR data. Clearly, the challenges of AutoExpert lie in the data-modality and annotation-task discrepancies. Meanwhile, publicly-available foundation models (FMs) serve as promising tools to tackle these challenges. Hence, we address AutoExpert by leveraging appropriate FMs within a conceptually simple pipeline, which (1) utilizes FMs for 2D object detection and segmentation in RGB images, (2) lifts 2D detections into 3D using known sensor poses, and (3) generates 3D cuboids for the 2D detections. In this pipeline, we progressively refine key components and eventually boost 3D detection mAP to 25.4, significantly higher than 12.1 achieved by prior arts.

3D检测自动标注激光雷达基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。