融合雷达与摄像头，提升自动驾驶3D目标检测精度。
Multi-Modal Sensor Fusion using Hybrid Attention for Autonomous Driving
- 用可变形注意力对齐雷达与视觉特征，实现跨模态融合
- 在VoD数据集上，融合模型性能优于单一传感器方案
- 支持近距离区域检测，适合实际自动驾驶场景
自动驾驶中的精准3D目标检测依赖互补传感器：摄像头提供密集语义但深度不可靠，毫米波雷达则具备精确的距离与速度测量能力，但几何信息稀疏。本文提出MMF-BEV框架，基于德夫特（View-of-Delft, VoD）4D雷达数据集，采用可变形注意力进行跨模态特征对齐。该框架包含两个分支：基于BEVDepth的摄像头分支和基于RadarBEVNet的雷达分支，均引入可变形自注意力机制，并通过可变形交叉注意力模块实现融合。评估了三种配置：仅摄像头、仅雷达、以及混合融合。通过传感器贡献分析量化不同距离下的模态权重，揭示了传感器间的互补性。采用两阶段训练策略——先以深度监督预训练摄像头分支，再联合训练雷达与融合模块，有效稳定学习过程。在VoD数据集上的实验表明，MMF-BEV始终优于单模态基线，且在全标注区域和近距感兴趣区域中，对所有物体类别均达到与先前融合方法相当甚至更优的性能。
原文摘要 · Abstract (English)
Accurate 3D object detection for autonomous driving requires complementary sensors. Cameras provide dense semantics but unreliable depth, while millimeter-wave radar offers precise range and velocity measurements with sparse geometry. We propose MMF-BEV, a radar-camera BEV fusion framework that leverages deformable attention for cross-modal feature alignment on the View-of-Delft (VoD) 4D radar dataset [1]. MMF-BEV builds a BEVDepth [2] camera branch and a RadarBEVNet [3] radar branch, each enhanced with Deformable Self-Attention, and fuses them via a Deformable Cross-Attention module. We evaluate three configurations: camera-only, radar-only, and hybrid fusion. A sensor contribution analysis quantifies per-distance modality weighting, providing interpretable evidence of sensor complementarity. A two-stage training strategy - pre-training the camera branch with depth supervision, then jointly training radar and fusion modules stabilizes learning. Experiments on VoD show that MMF-BEV consistently outperforms unimodal baselines and achieves competitive results against prior fusion methods across all object classes in both the full annotated area and near-range Region of Interest.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。