融合4D雷达与相机，提升复杂环境下的3D目标检测精度。
Depth-aware Fusion Method based on Image and 4D Radar Spectrum for 3D Object Detection
- 结合4D雷达谱与深度感知图像，在鸟瞰图下实现多模态融合
- 利用生成对抗网络从雷达数据重建深度图，弥补无深度传感器缺陷
- 在雨雪雾等恶劣条件下仍保持高鲁棒性，适合自动驾驶场景
自动驾驶的安全性与可靠性对公众接受度至关重要。为确保环境感知的准确与鲁棒，智能车辆需在各种环境下保持高性能。毫米波雷达具备强穿透能力，可在雨、雪、雾等恶劣天气下正常工作。传统3D毫米波雷达仅提供距离、多普勒和方位信息；尽管4D毫米波雷达增加了俯仰分辨率，但受恒虚警率（CFAR）处理影响，点云仍稀疏。相比之下，相机提供丰富的语义信息，但对光照与天气敏感。本文利用4D毫米波雷达与相机这两种互补且成本低的传感器，将4D雷达谱与深度感知图像结合，通过注意力机制在鸟瞰图（BEV）空间融合纹理丰富的图像与深度丰富的雷达数据，从而提升3D目标检测性能。此外，提出基于GAN的网络，从雷达谱生成深度图，弥补无深度传感器的不足，进一步提高检测精度。
原文摘要 · Abstract (English)
Safety and reliability are crucial for the public acceptance of autonomous driving. To ensure accurate and reliable environmental perception, intelligent vehicles must exhibit accuracy and robustness in various environments. Millimeter-wave radar, known for its high penetration capability, can operate effectively in adverse weather conditions such as rain, snow, and fog. Traditional 3D millimeter-wave radars can only provide range, Doppler, and azimuth information for objects. Although the recent emergence of 4D millimeter-wave radars has added elevation resolution, the radar point clouds remain sparse due to Constant False Alarm Rate (CFAR) operations. In contrast, cameras offer rich semantic details but are sensitive to lighting and weather conditions. Hence, this paper leverages these two highly complementary and cost-effective sensors, 4D millimeter-wave radar and camera. By integrating 4D radar spectra with depth-aware camera images and employing attention mechanisms, we fuse texture-rich images with depth-rich radar data in the Bird's Eye View (BEV) perspective, enhancing 3D object detection. Additionally, we propose using GAN-based networks to generate depth images from radar spectra in the absence of depth sensors, further improving detection accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。