arXiv:2411.08665cs.CV2024-11被引 11

用图像和地图融合定位,提升真实场景下的精度与泛化能力。

OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Fused Geometric and Semantic Guidance

  • 结合视觉与几何信息,通过深度适配器对齐单目图像与地图视角。
  • 在多个数据集上定位误差低于10米,跨区域、跨条件表现更优。
  • 适合需要高精度地图定位的自动驾驶与移动导航应用。

OpenStreetMap(OSM)作为丰富的志愿地理信息(VGI)来源,通过整合视觉观测与矢量化地图数据,支持人类自我定位与场景理解。然而,模态与视角差异导致图像与紧凑地图表示难以有效匹配,限制了VGI在真实定位中的潜力。受人脑融合几何与语义进行空间定位的启发,本文提出OSMLoc:一种基于第一人称图像的脑启发式视觉定位方法。该方法通过视觉基础模型提取强特征,设计几何引导的深度分布适配器,实现单目深度估计与相机到鸟瞰图(BEV)变换的对齐,并利用OSM语义嵌入作为辅助指导图像与地图特征匹配。为验证有效性,我们构建了一个全球范围的跨区域、跨条件(CC)基准测试集。在MGL、CC验证基准及KITTI数据集上的实验表明,本方法显著优于现有方法。代码、预训练模型、CC基准及更多结果详见:https://github.com/WHU-USI3DV/OSMLoc。

原文摘要 · Abstract (English)

OpenStreetMap (OSM), a rich and versatile source of volunteered geographic information (VGI), facilitates human self-localization and scene understanding by integrating nearby visual observations with vectorized map data. However, the disparity in modalities and perspectives poses a major challenge for effectively matching camera imagery with compact map representations, thereby limiting the full potential of VGI data in real-world localization applications. Inspired by the fact that the human brain relies on the fusion of geometric and semantic understanding for spatial localization tasks, we propose the OSMLoc in this paper. OSMLoc is a brain-inspired visual localization approach based on first-person-view images against the OSM maps. It integrates semantic and geometric guidance to significantly improve accuracy, robustness, and generalization capability. First, we equip the OSMLoc with the visual foundational model to extract powerful image features. Second, a geometry-guided depth distribution adapter is proposed to bridge the monocular depth estimation and camera-to-BEV transform. Thirdly, the semantic embeddings from the OSM data are utilized as auxiliary guidance for image-to-OSM feature matching. To validate the proposed OSMLoc, we collect a worldwide cross-area and cross-condition (CC) benchmark for extensive evaluation. Experiments on the MGL dataset, CC validation benchmark, and KITTI dataset have demonstrated the superiority of our method. Code, pre-trained models, CC validation benchmark, and additional results are available at: https://github.com/WHU-USI3DV/OSMLoc.

视觉定位地图匹配语义融合自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。