解决自动驾驶中相机与激光雷达特征错位问题,提升感知精度。
GraphBEV++: Multi-Modal Feature Alignment for Autonomous Driving

- 用图匹配构建局部深度特征,修正传感器间局部错位。
- 提出可变形与扩散两种全局对齐机制,分别适配不同架构。
- 在nuScenes、Waymo等数据集上显著提升长距离检测和占用预测性能。
BEV感知中的特征错位是自动驾驶中一个关键但常被忽视的挑战,尤其在激光雷达与摄像头校准不确定时更为突出。为此,我们提出稳健的多模态融合框架GraphBEV++,系统性缓解投影引起的错位问题。该框架包含两个核心模块:LocalAlign-v2和GlobalAlign-v2。LocalAlign-v2通过图匹配引入邻域感知的深度特征,修正局部错位,兼容基于LSS和查询的BEV表示,适用于BEVFusion和BEVFormer架构,实现跨范式一致对齐。GlobalAlign-v2包含两种变体:可变形与扩散。可变形变体显式学习跨模态特征偏移,解决基于LSS的多模态BEV全局错位;扩散变体则通过注入噪声模拟查询式BEV中的隐式错位,并利用去噪过程恢复对齐特征。实验表明,GraphBEV++在nuScenes和Waymo子集的错位噪声下达到当前最优性能,在Argoverse2上提升长距离检测能力,并有效泛化至3D占用预测任务,无论在干净或噪声环境下均显著提高估计精度与鲁棒性。此外,GraphBEV++在端到端自动驾驶中有效缓解错位问题。相比五种基线(UniAD、VAD、FusionAD、MomAD、WoTE),在开放环(nuScenes)和闭环(Bench2Drive和NAVSIM)评估中,于感知、预测与规划任务上均表现更优。
原文摘要 · Abstract (English)
Feature misalignment in BEV perception is a critical yet often overlooked challenge in autonomous driving, especially under calibration uncertainties between LiDAR and camera sensors. To address this issue, we propose a robust multi-modal fusion framework, GraphBEV++, which systematically mitigates projection-induced misalignment. The framework consists of two key modules: LocalAlign-v2 and GlobalAlign-v2. LocalAlign-v2 introduces neighborhood-aware depth features via graph matching to correct local misalignment. It supports both LSS-based and query-based BEV representations, making it compatible with BEVFusion and BEVFormer architectures for consistent cross-paradigm alignment. GlobalAlign-v2 encompasses two variants: Deformable and Diffusion. The Deformable variant addresses global misalignment in LSS-based multi-modal BEV by explicitly learning cross-modal feature offsets. In contrast, the Diffusion variant targets implicit misalignment in query-based BEV by injecting noise to simulate misalignment and employing a denoising process to recover aligned features. Experimental results show that GraphBEV++ achieves state-of-the-art performance under misalignment noise on nuScenes and Waymo subset, improves long-range detection on Argoverse2, and generalizes effectively to the 3D occupancy prediction task, consistently improving occupancy estimation accuracy and robustness under both clean and noisy settings. Furthermore, GraphBEV++ effectively alleviates misalignment issues in end-to-end autonomous driving. Compared with five baselines (UniAD, VAD, FusionAD, MomAD, and WoTE), it demonstrates superior performance in both open-loop (nuScenes) and closed-loop (Bench2Drive and NAVSIM) evaluations across perception, prediction, and planning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。