arXiv:2508.20530cs.CV2025-08中稿 · ACM MM 2025被引 2

通过早期融合激光雷达与图像数据,提升无监督3D目标检测的伪框质量。

Enhancing Pseudo-Boxes via Data-Level LiDAR-Camera Fusion for Unsupervised 3D Object Detection

  • 在数据层面融合激光雷达点云与图像,提升伪框生成精度。
  • 在nuScenes上达到28.4% mAP,显著优于现有方法。
  • 适合研究无监督3D检测、多模态融合的学者参考。

现有的基于激光雷达的3D目标检测器通常依赖人工标注标签以获得良好性能,但高质量3D标注耗时且费力。为解决此问题,近期工作尝试通过引入RGB图像作为辅助模态来实现无监督3D目标检测,辅助生成伪框。然而,这些方法仅简单地在标签层面融合由激光雷达点云和图像生成的伪框,未能充分利用两者在数据层面的互补性,导致伪框质量提升有限。为此,本文提出一种新颖的数据级融合框架,在早期阶段融合RGB图像与激光雷达数据。具体而言,利用视觉基础模型对图像进行实例分割和深度估计,并设计双向融合机制:真实点云从2D空间获取类别标签,2D像素投影至3D空间以增强点云密度。为抑制深度估计与分割结果中的噪声,提出局部与全局过滤方法——局部半径滤波抑制深度误差,全局统计滤波去除分割引起的离群点。此外,提出基于数据级融合的动态自进化策略,在密集表征下迭代优化伪框,显著提升定位精度。在nuScenes数据集上的大量实验表明,采用本方法训练的检测器在nuScenes验证集上达到28.4% mAP,显著优于此前最先进方法。

原文摘要 · Abstract (English)

Existing LiDAR-based 3D object detectors typically rely on manually annotated labels for training to achieve good performance. However, obtaining high-quality 3D labels is time-consuming and labor-intensive. To address this issue, recent works explore unsupervised 3D object detection by introducing RGB images as an auxiliary modal to assist pseudo-box generation. However, these methods simply integrate pseudo-boxes generated by LiDAR point clouds and RGB images. Yet, such a label-level fusion strategy brings limited improvements to the quality of pseudo-boxes, as it overlooks the complementary nature in terms of LiDAR and RGB image data. To overcome the above limitations, we propose a novel data-level fusion framework that integrates RGB images and LiDAR data at an early stage. Specifically, we utilize vision foundation models for instance segmentation and depth estimation on images and introduce a bi-directional fusion method, where real points acquire category labels from the 2D space, while 2D pixels are projected onto 3D to enhance real point density. To mitigate noise from depth and segmentation estimations, we propose a local and global filtering method, which applies local radius filtering to suppress depth estimation errors and global statistical filtering to remove segmentation-induced outliers. Furthermore, we propose a data-level fusion based dynamic self-evolution strategy, which iteratively refines pseudo-boxes under a dense representation, significantly improving localization accuracy. Extensive experiments on the nuScenes dataset demonstrate that the detector trained by our method significantly outperforms that trained by previous state-of-the-art methods with 28.4$\%$ mAP on the nuScenes validation benchmark.

3D检测多模态融合无监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。