融合雷达与摄像头，提升自动驾驶在恶劣环境下的感知与规划能力。
SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving
- 用稀疏3D特征对齐和多普勒速度估计，实现端到端融合。
- 在nuScenes等数据集上，检测、追踪、轨迹预测等多项指标显著提升。
- 适合追求高安全性的自动驾驶系统研发者参考。
端到端自动驾驶系统通过统一优化感知、运动预测和规划，有望实现更强性能。然而，纯视觉方法在恶劣天气、部分遮挡及精确速度估计方面存在根本性局限，这些是安全敏感场景中避免碰撞所必需的。为此，我们提出 SpaRC-AD，一种面向规划的查询式相机-雷达融合框架。通过稀疏3D特征对齐和基于多普勒的速度估计，实现高质量3D场景表征，用于优化目标锚点、地图折线和运动建模。在多个自动驾驶任务中，相比最先进的纯视觉基线，本方法取得显著提升：3D检测（mAP提升4.8%）、多目标跟踪（AMOTA提升8.3%）、在线地图构建（mAP提升1.8%）、运动预测（MADE降低4.0%）以及轨迹规划（L2距离减少0.1m，TPC降低9%）。在真实世界开放环nuScenes、长时程T-nuScenes及闭环仿真器Bench2Drive等多个挑战性基准上，实现了空间一致性与时间连续性。结果表明，雷达融合在需要精准运动理解与长时程轨迹预测的安全关键场景中具有显著有效性。所有实验代码已公开于 https://phi-wol.github.io/sparcad/
原文摘要 · Abstract (English)
End-to-end autonomous driving systems promise stronger performance through unified optimization of perception, motion forecasting, and planning. However, vision-based approaches face fundamental limitations in adverse weather conditions, partial occlusions, and precise velocity estimation - critical challenges in safety-sensitive scenarios where accurate motion understanding and long-horizon trajectory prediction are essential for collision avoidance. To address these limitations, we propose SpaRC-AD, a query-based end-to-end camera-radar fusion framework for planning-oriented autonomous driving. Through sparse 3D feature alignment, and doppler-based velocity estimation, we achieve strong 3D scene representations for refinement of agent anchors, map polylines and motion modelling. Our method achieves strong improvements over the state-of-the-art vision-only baselines across multiple autonomous driving tasks, including 3D detection (+4.8% mAP), multi-object tracking (+8.3% AMOTA), online mapping (+1.8% mAP), motion prediction (-4.0% mADE), and trajectory planning (-0.1m L2 and -9% TPC). We achieve both spatial coherence and temporal consistency on multiple challenging benchmarks, including real-world open-loop nuScenes, long-horizon T-nuScenes, and closed-loop simulator Bench2Drive. We show the effectiveness of radar-based fusion in safety-critical scenarios where accurate motion understanding and long-horizon trajectory prediction are essential for collision avoidance. The source code of all experiments is available at https://phi-wol.github.io/sparcad/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。