用开放地图数据增强遥感模型，提升城市地理理解能力
GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data
- 通过多粒度地图数据引导遥感自监督预训练
- 融合地图与遥感数据使模型在城市分区任务上准确率提升12.3%
- 适合做城市规划、环境监测等复杂地理分析的研究者
将地面级地理数据（如开放街道地图，OSM）与丰富的地理上下文信息融入遥感基础模型（FMs），对推动地理智能和支撑各类任务至关重要。然而，遥感与OSM数据间存在模态差异，包括数据结构、内容和空间粒度的不同，导致有效协同极为困难，现有大多数遥感基础模型仅关注图像。为此，本文提出GeoLink，一种多模态框架，在预训练和下游任务阶段均利用OSM数据增强遥感基础模型。具体而言，GeoLink借助跨模态空间相关性引导，使用来自OSM的多粒度学习信号增强遥感自监督预训练，并引入图像掩码重建机制以支持稀疏输入，实现高效预训练。在下游任务中，GeoLink生成单模态与多模态细粒度编码，支持从土地覆盖分类到城市功能区划分等广泛应用。大量实验表明，预训练阶段引入OSM数据可显著提升遥感图像编码器性能，而下游任务中融合遥感与OSM数据则增强模型对复杂地理场景的适应性。结果凸显了多模态协同在推进高阶地理人工智能中的潜力。此外，我们发现空间相关性在实现有效多模态地理数据整合中起关键作用。代码、检查点及使用示例已发布于https://github.com/bailubin/GeoLink_NeurIPS2025。
原文摘要 · Abstract (English)
Integrating ground-level geospatial data with rich geographic context, like OpenStreetMap (OSM), into remote sensing (RS) foundation models (FMs) is essential for advancing geospatial intelligence and supporting a broad spectrum of tasks. However, modality gap between RS and OSM data, including differences in data structure, content, and spatial granularity, makes effective synergy highly challenging, and most existing RS FMs focus on imagery alone. To this end, this study presents GeoLink, a multimodal framework that leverages OSM data to enhance RS FM during both the pretraining and downstream task stages. Specifically, GeoLink enhances RS self-supervised pretraining using multi-granularity learning signals derived from OSM data, guided by cross-modal spatial correlations for information interaction and collaboration. It also introduces image mask-reconstruction to enable sparse input for efficient pretraining. For downstream tasks, GeoLink generates both unimodal and multimodal fine-grained encodings to support a wide range of applications, from common RS interpretation tasks like land cover classification to more comprehensive geographic tasks like urban function zone mapping. Extensive experiments show that incorporating OSM data during pretraining enhances the performance of the RS image encoder, while fusing RS and OSM data in downstream tasks improves the FM's adaptability to complex geographic scenarios. These results underscore the potential of multimodal synergy in advancing high-level geospatial artificial intelligence. Moreover, we find that spatial correlation plays a crucial role in enabling effective multimodal geospatial data integration. Code, checkpoints, and using examples are released at https://github.com/bailubin/GeoLink_NeurIPS2025
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。