用2D图像标签训练3D地图,低成本实现自动驾驶高精地图在线构建
MapRF: Weakly Supervised Online HD Map Construction via NeRF-Guided Self-Training
- 基于NeRF生成一致的3D几何与语义伪标签,实现弱监督学习
- 自训练机制使模型性能达全监督方法的75%,超越其他仅用2D标签的方法
- 适合需要低成本、可扩展地图构建的自动驾驶系统研发者
自动驾驶系统依赖高精地图提供道路基础设施的关键信息。在线构建高精地图可通过车载传感器实现规模化局部地图生成。然而,现有方法通常依赖昂贵的3D地图标注进行训练,限制了其在多样化驾驶环境中的泛化与可扩展性。本文提出MapRF,一种弱监督框架,仅使用2D图像标签即可学习构建3D地图。为生成高质量伪标签,引入一种基于地图预测条件的新型神经辐射场(NeRF)模块,重建视图一致的3D几何与语义。这些伪标签被迭代用于自训练,以无额外监督的方式逐步提升地图网络性能。此外,为缓解自训练过程中的误差累积,提出地图到射线匹配策略,将地图预测与来自2D标签的相机射线对齐。在Argoverse 2和nuScenes数据集上的大量实验表明,MapRF性能接近全监督基线,达到其约75%水平,同时优于多个仅使用2D标签的方法。这凸显了MapRF在实现可扩展、低成本的自动驾驶在线高精地图构建方面的潜力。
原文摘要 · Abstract (English)
Autonomous driving systems benefit from high-definition (HD) maps that provide critical information about road infrastructure. The online construction of HD maps offers a scalable approach to generate local maps from on-board sensors. However, existing methods typically rely on costly 3D map annotations for training, which limits their generalization and scalability across diverse driving environments. In this work, we propose MapRF, a weakly supervised framework that learns to construct 3D maps using only 2D image labels. To generate high-quality pseudo labels, we introduce a novel Neural Radiance Fields (NeRF) module conditioned on map predictions, which reconstructs view-consistent 3D geometry and semantics. These pseudo labels are then iteratively used to refine the map network in a self-training manner, enabling progressive improvement without additional supervision. Furthermore, to mitigate error accumulation during self-training, we propose a Map-to-Ray Matching strategy that aligns map predictions with camera rays derived from 2D labels. Extensive experiments on the Argoverse 2 and nuScenes datasets demonstrate that MapRF achieves performance comparable to fully supervised methods, attaining around 75% of the baseline while surpassing several approaches using only 2D labels. This highlights the potential of MapRF to enable scalable and cost-effective online HD map construction for autonomous driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。