用激光雷达生成伪雷达数据,提升雷达相机融合检测精度。
CLLAP: Contrastive Learning-based LiDAR-Augmented Pretraining for Enhanced Radar-Camera Fusion

- 用激光雷达数据生成伪雷达数据,解决标注雷达数据稀缺问题。
- 在NuScenes和Lyft数据集上,三个基线模型检测精度显著提升。
- 可即插即用,适合需要增强雷达相机融合的自动驾驶研究者。
精准的3D目标检测对自动驾驶至关重要,需可靠且成本低的传感器在恶劣天气下工作。摄像头与毫米波雷达融合成为有前景的解决方案,但现有方法常依赖精细标注的雷达数据,这类数据稀缺且标注成本高。为此,我们提出CLLAP——基于对比学习的激光雷达增强预训练框架,以提升现有雷达-相机融合模型在3D目标检测中的性能。CLLAP利用丰富的激光雷达数据,通过提出的L2R(LiDAR-to-Radar)采样方法生成伪雷达数据,并引入新颖的双阶段、双模态对比学习策略,实现从成对伪雷达图像数据中进行有效自监督学习。该方法可无缝集成到现有雷达-相机融合模型中,以即插即用方式增强特征提取能力,提升检测准确率与鲁棒性。在NuScenes和Lyft Level 5数据集上的实验表明,三种基线模型均获得显著性能提升,验证了CLLAP在推动雷达-相机融合方面的有效性。
原文摘要 · Abstract (English)
Accurate 3D object detection is critical for autonomous driving, necessitating reliable, cost-effective sensors capable of operating in adverse weather conditions. Camera and millimeter-wave radar fusion has emerged as a promising solution; however, these methods often rely on finely annotated radar data, which is scarce and labor-intensive to produce. To address this challenge, we present CLLAP, a Contrastive Learning-based LiDAR-Augmented Pretraining framework that enhances the performance of existing radar-camera fusion methods for 3D object detection. CLLAP leverages abundant LiDAR data to generate pseudo-radar data using the proposed L2R (LiDAR-to-Radar) Sampling method. Then, it incorporates this data into a novel dual-stage, dual-modality contrastive learning strategy, enabling effective self-supervised learning from paired pseudo-radar and image data. This approach facilitates effective pretraining of existing radar-camera fusion models in a plug-and-play manner, enhancing their feature extraction capabilities and improving detection accuracy and robustness. Experimental results using NuScenes and Lyft Level 5 datasets demonstrate significant performance improvements across three baseline models, highlighting CLLAP's effectiveness in advancing radar-camera fusion for autonomous driving applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。