arXiv:2511.01082cs.CVcs.AI2025-11中稿 · IEEE International…被引 5

用分层令牌预测法,像猜谜一样逐步缩小图片的地理位置范围。

GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction

  • 模仿人类从大区到具体地址的推理方式,逐级预测地理网格单元。
  • 在Im2GPS3k和YFCC4k数据集上,精度比现有方法最高提升13.9%。
  • 适合需要高精度图像定位的应用,如地理信息检索与智能导航。

图像地理定位任务旨在确定图像的地理来源,面临视觉相似性跨区域及搜索空间庞大的挑战。本文提出一种受人类推理启发的分层序列预测方法:通过S2网格(一种嵌套多分辨率全球网格)逐级预测更精细的地理单元,条件依赖于视觉输入与先前预测结果,模拟自回归文本生成过程。模型在推理阶段采用多种自顶向下遍历策略,结合束搜索与多样本推理,有效管理不确定性并探索多条可能路径。在Im2GPS3k与YFCC4k数据集上评估,不依赖多模态大模型时,性能超越多数基线,最高准确率提升13.9%;引入多模态大模型后,全面领先所有对比方法,刷新各指标纪录。

原文摘要 · Abstract (English)

Image geolocalization, the task of determining an image's geographic origin, poses significant challenges, largely due to visual similarities across disparate locations and the large search space. To address these issues, we propose a hierarchical sequence prediction approach inspired by how humans narrow down locations from broad regions to specific addresses. Analogously, our model predicts geographic tokens hierarchically, first identifying a general region and then sequentially refining predictions to increasingly precise locations. Rather than relying on explicit semantic partitions, our method uses S2 cells, a nested, multiresolution global grid, and sequentially predicts finer-level cells conditioned on visual inputs and previous predictions. This procedure mirrors autoregressive text generation in large language models. Much like in language modeling, final performance depends not only on training but also on inference-time strategy. We investigate multiple top-down traversal methods for autoregressive sampling, incorporating techniques from test-time compute scaling used in language models. Specifically, we integrate beam search and multi-sample inference while exploring various selection strategies to determine the final output. This enables the model to manage uncertainty by exploring multiple plausible paths through the hierarchy. We evaluate our method on the Im2GPS3k and YFCC4k datasets against two distinct sets of baselines: those that operate without a Multimodal Large Language Model (MLLM) and those that leverage one. In the MLLM-free setting, our model surpasses other comparable baselines on nearly all metrics, achieving state-of-the-art performance with accuracy gains of up to 13.9%. When augmented with an MLLM, our model outperforms all baselines, setting a new state-of-the-art across all metrics. The source code is available at https://github.com/NNargesNN/GeoToken.

图像定位分层预测S2网格自回归

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。