arXiv:2409.14751cs.CVcs.AI2024-09ICRA被引 11

融合雷达与视觉信息,提升3D目标检测精度与鲁棒性

UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection

  • 引入雷达深度生成模块,更好利用雷达特有信息
  • 统一特征融合框架,实现多模态共享特征提取
  • 在极端条件下仍保持性能,适合自动驾驶场景

4D毫米波雷达能提供高度信息和密集的三维点云数据,在3D目标检测中日益流行。近年来,雷达-视觉融合模型性能已接近激光雷达基线模型,具备硬件成本低、恶劣环境下更稳定的优势。但多数融合模型将雷达视为稀疏激光雷达,未能充分利用其特性;且多模态网络对单一模态失效敏感,尤其依赖视觉。为此,本文提出雷达深度升维-投射-射击(RDL)模块,将雷达特有信息融入深度预测过程,提升视觉鸟瞰图(BEV)特征质量。进一步提出统一特征融合(UFF)方法,通过共享模块跨模态提取BEV特征。为评估模型鲁棒性,设计新型失败测试(FT)消融实验,通过注入高斯噪声模拟视觉模态失效。在View-of-Delft(VoD)和TJ4D数据集上进行大量实验。结果表明,所提统一BEVFusion(UniBEVFusion)网络在TJ4D数据集上显著优于现有最佳模型,3D检测准确率提升3.96%,BEV检测准确率提升4.17%。

原文摘要 · Abstract (English)

4D millimeter-wave (MMW) radar, which provides both height information and dense point cloud data over 3D MMW radar, has become increasingly popular in 3D object detection. In recent years, radar-vision fusion models have demonstrated performance close to that of LiDAR-based models, offering advantages in terms of lower hardware costs and better resilience in extreme conditions. However, many radar-vision fusion models treat radar as a sparse LiDAR, underutilizing radar-specific information. Additionally, these multi-modal networks are often sensitive to the failure of a single modality, particularly vision. To address these challenges, we propose the Radar Depth Lift-Splat-Shoot (RDL) module, which integrates radar-specific data into the depth prediction process, enhancing the quality of visual Bird-Eye View (BEV) features. We further introduce a Unified Feature Fusion (UFF) approach that extracts BEV features across different modalities using shared module. To assess the robustness of multi-modal models, we develop a novel Failure Test (FT) ablation experiment, which simulates vision modality failure by injecting Gaussian noise. We conduct extensive experiments on the View-of-Delft (VoD) and TJ4D datasets. The results demonstrate that our proposed Unified BEVFusion (UniBEVFusion) network significantly outperforms state-of-the-art models on the TJ4D dataset, with improvements of 1.44 in 3D and 1.72 in BEV object detection accuracy.

3D检测雷达融合多模态自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。