用视觉语言模型引导检索,实现全球尺度精准定位。
VLM-Guided Visual Place Recognition for Planet-Scale Geo-Localization
- 先用VLM生成地理先验,缩小检索范围
- 在街级和城市级定位上分别提升4.51%和13.52%
- 适合需要高精度与可解释性的导航系统
从单张图像实现全球尺度的地理定位(本质上是极端化的“被劫持机器人”问题)是导航、自动驾驶和灾后响应中的基础且极具挑战的任务,受限于地点多样性、环境变化与场景差异。传统基于检索的方法存在可扩展性差和感知混淆问题,而基于分类的方法泛化能力弱且需大量训练数据。近期视觉语言模型(VLMs)通过上下文理解与推理提供了新思路,但常出现幻觉且缺乏可解释性,难以独立使用。本文提出一种融合VLM与检索式视觉定位(VPR)的混合框架:首先利用VLM生成地理先验,有效约束检索空间;随后执行检索并采用重排序机制,依据特征相似性和初始坐标邻近性筛选最合理的匹配结果。在多个地理定位基准测试中,该方法持续优于现有最先进方法,尤其在街级(最高提升4.51%)和城市级(最高提升13.52%)表现显著。结果表明,结合VLM生成的地理先验与VPR能构建可扩展、鲁棒且高精度的地理定位系统。
原文摘要 · Abstract (English)
Geo-localization from a single image at planet scale (essentially an advanced or extreme version of the kidnapped robot problem) is a fundamental and challenging task in applications such as navigation, autonomous driving and disaster response due to the vast diversity of locations, environmental conditions, and scene variations. Traditional retrieval-based methods for geo-localization struggle with scalability and perceptual aliasing, while classification-based approaches lack generalization and require extensive training data. Recent advances in vision-language models (VLMs) offer a promising alternative by leveraging contextual understanding and reasoning. However, while VLMs achieve high accuracy, they are often prone to hallucinations and lack interpretability, making them unreliable as standalone solutions. In this work, we propose a novel hybrid geo-localization framework that combines the strengths of VLMs with retrieval-based visual place recognition (VPR) methods. Our approach first leverages a VLM to generate a prior, effectively guiding and constraining the retrieval search space. We then employ a retrieval step, followed by a re-ranking mechanism that selects the most geographically plausible matches based on feature similarity and proximity to the initially estimated coordinates. We evaluate our approach on multiple geo-localization benchmarks and show that it consistently outperforms prior state-of-the-art methods, particularly at street (up to 4.51%) and city level (up to 13.52%). Our results demonstrate that VLM-generated geographic priors in combination with VPR lead to scalable, robust, and accurate geo-localization systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。