用地图扩散提升视觉与高精地图融合,让自动驾驶更准更稳。
DiffSemanticFusion: Semantic Raster BEV Fusion for Autonomous Driving via Online HD Map Diffusion
- 通过语义栅格融合与地图扩散模块,结合视觉与高精地图优势。
- 在nuScenes预测任务上提升5.1%,NAVSIM端到端驾驶提升15%。
- 适合做轨迹预测和端到端自动驾驶的团队,可无缝接入现有系统。
自动驾驶需要精准的场景理解,包括道路几何、交通参与者及其语义关系。在在线高精地图生成场景中,基于栅格的表示适合视觉模型但缺乏几何精度,而基于图的表示虽保留结构细节,但在缺乏精确地图时易不稳定。为此,我们提出DiffSemanticFusion——一种用于多模态轨迹预测与规划的融合框架。该方法在语义栅格融合的鸟瞰图空间中推理,并引入地图扩散模块,提升在线高精地图表示的稳定性和表达能力。我们在两个下游任务上验证框架:轨迹预测与面向规划的端到端自动驾驶。在真实世界基准nuScenes和NAVSIM上的实验表明,性能优于多个先进方法。在nuScenes预测任务中,将DiffSemanticFusion与在线地图感知的QCNet结合,实现5.1%的性能提升;在NAVSIM端到端自动驾驶中,于NavHard场景下取得15%的性能增益。大量消融与敏感性分析显示,地图扩散模块可无缝集成至其他向量类方法以提升性能。所有代码与资源见https://github.com/SunZhigang7/DiffSemanticFusion。
原文摘要 · Abstract (English)
Autonomous driving requires accurate scene understanding, including road geometry, traffic agents, and their semantic relationships. In online HD map generation scenarios, raster-based representations are well-suited to vision models but lack geometric precision, while graph-based representations retain structural detail but become unstable without precise maps. To harness the complementary strengths of both, we propose DiffSemanticFusion -- a fusion framework for multimodal trajectory prediction and planning. Our approach reasons over a semantic raster-fused BEV space, enhanced by a map diffusion module that improves both the stability and expressiveness of online HD map representations. We validate our framework on two downstream tasks: trajectory prediction and planning-oriented end-to-end autonomous driving. Experiments on real-world autonomous driving benchmarks, nuScenes and NAVSIM, demonstrate improved performance over several state-of-the-art methods. For the prediction task on nuScenes, we integrate DiffSemanticFusion with the online HD map informed QCNet, achieving a 5.1\% performance improvement. For end-to-end autonomous driving in NAVSIM, DiffSemanticFusion achieves state-of-the-art results, with a 15\% performance gain in NavHard scenarios. In addition, extensive ablation and sensitivity studies show that our map diffusion module can be seamlessly integrated into other vector-based approaches to enhance performance. All artifacts are available at https://github.com/SunZhigang7/DiffSemanticFusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。