提出可靠性感知融合框架,提升恶劣天气下多传感器3D检测精度
RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather

- 通过显式监督像素级可靠性,动态识别并抑制不可靠视觉信息
- 在K-Radar和VoD数据集上实现最高+6.5 APBEV和+7.4 AP3D提升
- 仅训练新增相机分支与融合模块,兼容预训练激光雷达-雷达模型
恶劣天气下鲁棒的3D目标检测面临传感器性能下降的挑战。尽管激光雷达与4D雷达融合具有潜力,但其点云稀疏性在雨雪等条件下更加严重,导致物体回波缺失。摄像头虽能提供视觉线索,但在雾、雪或雨中也因镜头遮挡而部分失效。现有图像融合方法依赖检测损失学习置信图,缺乏直接监督。本文提出可靠性感知融合(RAF),显式监督每像素可靠性估计,为不可靠视觉信息的识别与抑制提供直接学习信号。框架复用预训练的激光雷达-雷达网络,仅训练新增的相机分支、鸟瞰图融合编码器及检测头。在K-Radar和VoD数据集上的大量实验表明,RAF持续优于激光雷达-雷达基线,实现最高+6.5 APBEV和+7.4 AP3D增益。代码已开源。
原文摘要 · Abstract (English)
Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5 $AP_{BEV}$ and +7.4 $AP_{3D}$ gains. Code is available at https://github.com/parkie0517/RAF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。