用网页反向图像搜索增强全球定位，提升未知场景的准确性
GeoSearch: Augmenting Worldwide Geolocalization with Web-Scale Reverse Image Search and Image Matching
- 将网络级反向图像搜索融入大模型推理流程
- 在Im2GPS3k和YFCC4k上超越现有方法，尤其在无参考场景表现优异
- 适合做开放世界地理定位研究或需要高精度定位的开发者
全球图像地理定位旨在预测任意地球图像的经纬度，但受全球视觉多样性影响仍具挑战。现有基于检索增强生成（RAG）与大模态模型（LMMs）的方法依赖固定数据库中的候选样本进行推理，对未见场景表现不佳。本文提出GeoSearch，一种开放世界的地理定位框架，将网络规模的反向图像搜索集成到RAG流程中。该方法通过数据库检索的坐标与网页提取的文本证据增强LMM提示。为降低无关内容带来的噪声，引入两层过滤机制：先进行图像匹配，再通过置信度门控筛选。在标准基准Im2GPS3k和YFCC4k上的泄漏感知评估表明，GeoSearch显著优于现有方法。代码与数据已公开，支持可复现性。
原文摘要 · Abstract (English)
Worldwide image geolocalization, which aims to predict the GPS coordinates of any image on Earth, remains challenging due to global visual diversity. Recent generative approaches based on Retrieval-Augmented Generation (RAG) and Large Multimodal Models (LMMs) leverage candidates retrieved from fixed databases for reasoning, but often struggle with scenes that are absent from the reference set. In this work, we propose GeoSearch, an open-world geolocation framework that integrates web-scale reverse image search into the RAG pipeline. GeoSearch augments LMM prompts with database-retrieved coordinates and textual evidence extracted from web pages. To mitigate noise from irrelevant content, we introduce a two-layer filtering mechanism consisting of image matching, followed by confidence-based gating. Experiments on standard benchmarks Im2GPS3k and YFCC4k demonstrate the superiority of GeoSearch under leakage-aware evaluation. Our code and data are publicly available to support reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。