融合卫星图与平面地图，显著提升跨视角定位精度
Fusing Satellite Imagery and Planimetric Maps for Cross-View Localization
- 设计跨模态条件模块与像素级融合规则，动态结合两种图像
- 在标准方法上引入融合模块后，平均定位误差降低30.13%
- 适合需要高精度地理定位的自动驾驶与地图应用
当前跨视角定位方法主要依赖卫星图像作为空中视图。尽管近期研究探索了平面地图（如OpenStreetMap瓦片），但性能仍显不足。然而，两者均广泛可用且具有互补优势：卫星图像更接近地面相机图像，细节更丰富；而平面地图包含标注对象（如路灯），在植被遮挡等区域仍具信息量。尽管已有单一工作提出端到端融合方法，但未在先进模型中验证其潜力。为此，我们提出一种新型融合模块，可增强标准编码器，并证明将卫星图像与平面地图结合能显著提升现有单模态方法性能。该模块包含：(i) 跨模态条件机制，使每种模态的编码感知另一模态；(ii) 块级融合规则，控制信息交换粒度。实验表明，该方法实现最先进结果，平均定位误差降低30.13%。定性分析显示，融合机制能自适应选择更优模态，整体精度提升。
原文摘要 · Abstract (English)
Current cross-view localization methods predominantly rely on satellite imagery as the aerial modality. Although recent work explores planimetric maps (e.g., OpenStreetMap tiles), these approaches often lag in performance. Yet both modalities are widely available and possess complementary properties. Satellite images are closer to ground-level camera imagery, offering finer detail, whereas planimetric maps contain annotated objects (e.g., streetlamps) and remain informative in areas where the ground is occluded, such as by foliage. Despite this, only one prior work provides an end-to-end method to fuse the two modalities, and it does not demonstrate their potential within state-of-the-art methods. To combine the strengths of both modalities, we propose a new fusion module that augments standard encoders and demonstrates that integrating satellite imagery with planimetric maps improves state-of-the-art single-modality methods. The module comprises (i) cross-modal conditioning, which processes each modality's encoding with awareness of the other, and (ii) a patch-level fusion rule that controls the granularity of information exchange. We achieve state-of-the-art results, reducing the mean localization error by 30.13\%. Qualitatively, the fusion adaptively selects the more informative modality, improving overall accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。