让地图定位模型能说出判断依据,提升可解释性。
Towards Interpretable Geo-localization: a Concept-Aware Global Image-GPS Alignment Framework
- 引入概念感知模块,将图像与位置嵌入对齐到地理概念空间
- 在多个数据集上精度超越GeoCLIP,且支持多任务泛化
- 首次实现地理定位的可解释性,适合需要决策透明的场景
全球地理定位旨在确定全球范围内拍摄图像的精确地理位置,通常依赖气候、地标和建筑风格等地理线索。尽管像GeoCLIP这样的模型通过对比学习实现了图像与位置的对齐并取得较高精度,但其可解释性仍不足。现有基于概念的解释方法难以与地理对齐目标协同优化,导致解释效果不佳。为此,我们提出一种融合全局地理定位与概念瓶颈的新框架,引入概念感知对齐模块,将图像和位置嵌入联合投影至一组地理概念(如热带气候、山脉、大教堂)构成的共享空间,并最小化概念级损失,从而在特定语义子空间中增强对齐,实现强可解释性。据我们所知,这是首个将可解释性引入地理定位的工作。大量实验表明,该方法在地理定位准确率上超过GeoCLIP,同时在多种地理空间预测任务中性能提升,揭示了更丰富的地理决策语义信息。
原文摘要 · Abstract (English)
Worldwide geo-localization involves determining the exact geographic location of images captured globally, typically guided by geographic cues such as climate, landmarks, and architectural styles. Despite advancements in geo-localization models like GeoCLIP, which leverages images and location alignment via contrastive learning for accurate predictions, the interpretability of these models remains insufficiently explored. Current concept-based interpretability methods fail to align effectively with Geo-alignment image-location embedding objectives, resulting in suboptimal interpretability and performance. To address this gap, we propose a novel framework integrating global geo-localization with concept bottlenecks. Our method inserts a Concept-Aware Alignment Module that jointly projects image and location embeddings onto a shared bank of geographic concepts (e.g., tropical climate, mountain, cathedral) and minimizes a concept-level loss, enhancing alignment in a concept-specific subspace and enabling robust interpretability. To our knowledge, this is the first work to introduce interpretability into geo-localization. Extensive experiments demonstrate that our approach surpasses GeoCLIP in geo-localization accuracy and boosts performance across diverse geospatial prediction tasks, revealing richer semantic insights into geographic decision-making processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。