提出高效鲁棒的4D雷达-相机融合框架,实现360°全景3D目标检测。
Sparse4D-Radar: An Efficient and Robust Framework for Surround-View 3D Object Detection via 4D Radar-Camera Fusion

- 通过稀疏查询融合雷达与相机特征,构建轻量级检测基线。
- 在复杂场景下提升7% mAP和10% ODS,推理速度接近10 FPS。
- 适合需要高精度、强鲁棒性的自动驾驶全景感知系统。
近年来,4D成像雷达因其在恶劣天气下的鲁棒性及输出目标速度的能力,在自动驾驶中受到广泛关注。然而,主流的4D雷达-相机融合方法仅支持前视感知,缺乏成熟的全景感知解决方案。直接将此类方法扩展至360°覆盖会带来巨大计算开销,限制实际部署。为此,本文提出Sparse4D-Radar,一个高效且鲁棒的全景多模态融合框架。首先设计可变形融合模块,将雷达-相机特征嵌入稀疏查询,构建轻量级版本Sparse4D-Radar-Base。进一步引入两个专用模块:基于雷达速度线索的运动感知机制Velocity-Consistency Sampling(VCS),以及根据特征置信度动态调整跨模态融合权重的自适应模态门控(AMG)。结合所有组件,构建高精度版本Sparse4D-Radar-Acc。在OmniHD-Scenes数据集上的实验表明,该方法在复杂驾驶场景下达到当前最优的全景3D检测性能,相比先前方法提升超过7% mAP和10% ODS,同时保持近10 FPS的推理速度,实现了检测精度、环境鲁棒性与推理效率的良好平衡。开源代码已发布于https://github.com/Aiuan/Sparse4D-Radar。
原文摘要 · Abstract (English)
In recent years, 4D imaging radar has gained wide attention in autonomous driving for its robustness against harsh weather and ability to output target velocity. Nevertheless, mainstream 4D radar-camera fusion methods only support front-view perception, lacking mature solutions for surround-view sensing. Directly expanding these pipelines to full 360° coverage introduces excessive computation cost and limits real-world deployment. To tackle these limitations, this work proposes Sparse4D-Radar, an efficient robust surround-view multi-modal fusion framework. We first design a Deformable Fusion module to embed radar-camera features into sparse queries, constructing the lightweight base version Sparse4D-Radar-Base. Two dedicated modules are further introduced to boost localization accuracy and modality stability: Velocity-Consistency Sampling (VCS) refines features via radar velocity cues for motion awareness, and Adaptive Modality Gating (AMG) dynamically adjusts cross-modal fusion weights according to feature confidence. Combining all components, we build Sparse4D-Radar-Acc for high-precision detection demands. Comprehensive experiments on OmniHD-Scenes verify that our approach achieves state-of-the-art surround-view 3D detection performance. Compared with prior arts, our method obtains over 7% mAP and 10% ODS improvements under complex driving scenes while running at nearly 10 FPS, striking a favorable trade-off among detection accuracy, environmental robustness and inference efficiency. Our open-source code is available at https://github.com/Aiuan/Sparse4D-Radar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。