arXiv:2607.05207cs.CVcs.LG2026-07

用环境数据探查遥感自监督模型的表征,发现其保留了真实物理关联。

Probing Geospatial SSL Representations with Environmental Signals

论文配图:Probing Geospatial SSL Representations with Environmental Signals
图 1 · 摘自论文原文
  • 用气象再分析数据探测自监督模型对环境变量的编码能力
  • 模型线性可访问的环境信号与下游任务表现相关
  • 适合关注遥感表征物理意义的研究者

自监督学习(SSL)旨在学习通用、可迁移的表征,而非针对单一任务优化。现有地理空间基准主要通过下游任务评估表征,难以揭示表征本身包含的信息。本文提出新问题:遥感图像的SSL表征是否保留了与成像过程共变的环境变量之间的统计关联?为此,我们使用与影像同位置的全球再分析数据集ERA5中的温度、降水、地表太阳辐射、地表气压和土壤体积含水量等物理一致变量进行探测。这些变量虽未用于预训练,但与哨兵1号和哨兵2号的光谱反射率和雷达后向散射有物理关联。我们结合内在表征度量,分析表征几何结构与下游性能及环境信号编码的关系。在相同条件下训练的DINO、MAE和MoCo模型显示,表征级指标能区分下游性能相似的模型,提供任务评估之外的补充信息。此外,环境信号的线性可访问性与PANGAEA基准中环境依赖任务的表现相关。最后,我们公开了与SSL4EO数据集对齐的ERA5标注,支持未来地理空间基础模型的物理驱动评估。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) is designed to learn generic, transferable representations rather than representations optimized for a single task. Most geospatial benchmarks evaluate representations solely through downstream tasks, providing limited insight into the information encoded within the representation itself. We ask a different question: do SSL representations of satellite imagery preserve statistical associations with environmental variables that co-vary with the imaging process? To answer this question, we probe SSL representations using co-located ERA5 reanalysis variables, a global dataset of physically consistent environmental variables, including temperature, precipitation, surface solar radiation, surface pressure, and volumetric soil water. These variables are physically related to the spectral reflectance and radar backscatter recorded by Sentinel-1 and Sentinel-2, making them meaningful evaluation targets despite not being used during SSL pretraining. We complement this probing analysis with intrinsic representation metrics to characterize representation geometry and investigate how these properties relate to downstream performance and the encoding of environmental signals. Using DINO, MAE, and MoCo models trained under identical conditions, we show that representation-level metrics distinguish models with similar downstream benchmark performance, providing complementary information beyond task-driven benchmarks. We further find that the linear accessibility of environmental signals is associated with performance on environmentally dependent tasks in the PANGAEA benchmark. Finally, we release ERA5 annotations co-located with the SSL4EO dataset to enable physically grounded representation evaluation for future geospatial foundation models.

自监督学习遥感表征环境信号地理空间

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。