用地理隔离度排序遥感图像，高效实现自监督预训练。
Far from the Crowd: Scalable Self-Supervised Learning via Geographic Isolation
- 基于地理位置信息构建无标签样本排序策略，无需解码或标注。
- 仅用20%~40%训练预算达到基线性能，最高提升5 mAP。
- 适合大规模遥感数据预训练，显著降低计算成本。
遥感图像自监督预训练通常将所有样本视为同等重要，但地理与视觉结构差异巨大。本文提出一种课程学习策略，通过地理隔离度（完全依赖已有地理位置元数据）对样本排序，无需图像解码、模型反馈或人工标注。该方法在对比和重建目标下均具可扩展性（复杂度为O(D log D)），并集成至MoCoV2和MAE中。在CopernicusBench的三个下游任务（BigEarthNet、DFC-2020、LCZ）上验证，仅需20%训练预算（MAE）或最多40%（MoCo）即可达到基线性能，最大提升5 mAP，各基准增益1-5点。相比视觉复杂度方案，预处理耗时减少140倍以上（4秒 vs. 568秒，SSL4EO）。CKA与有效秩分析显示，课程训练使编码器嵌入空间维度更高、利用更均匀。
原文摘要 · Abstract (English)
Self-supervised pretraining on remote sensing imagery typically treats all samples as equally informative, despite large variability in geographic and visual structure. We propose a curriculum learning strategy for self-supervised Earth observation that ranks samples by geographic isolation, a label-free proxy derived entirely from geolocation metadata already present in geospatial datasets, requiring no image decoding, no model feedback, and no manual annotation. Unlike visual complexity proxies, it scales as O(D log D) with dataset size D and is well-defined for both contrastive and reconstructive objectives. We integrate the proposed measure into MoCoV2 and MAE pretraining and evaluate across three downstream tasks from CopernicusBench (BigEarthNet, DFC-2020, LCZ). Our curriculum reaches baseline final-epoch performance using as few as 20% of the training budget (MAE) and at most 40% (MoCo) of the training budget, and improves final downstream performance by up to +5 mAP on BigEarthNet, with gains of 1-5 points across benchmarks, matching visual-complexity curricula while reducing pre-computation cost by more than 140x (4 s vs. 568 s on SSL4EO). A CKA and effective-rank analysis further reveals that curriculum-trained encoders develop higher-dimensional, more uniformly utilized embedding spaces throughout training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。