TerraFM统一融合雷达与光学遥感数据,提升全球地表识别能力。
TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation
- 自监督学习框架融合哨兵1/2多源遥感影像,用空间块和土地覆盖采样增强覆盖。
- 在GEO-Bench和Copernicus-Bench上超越已有模型,分类与分割表现更优。
- 适合遥感、环境监测、地理信息系统研究者使用,支持开源复现。
现代地球观测日益依赖深度学习挖掘卫星影像在多传感器与区域间的规模与多样性。尽管近期基础模型在地球观测任务中展现出良好泛化能力,但多数受限于训练数据的规模、地理覆盖范围及光谱多样性,这些因素对学习全局可迁移表征至关重要。本文提出TerraFM,一种可扩展的自监督学习模型,利用全球分布的哨兵-1与哨兵-2影像,结合大尺寸空间块和土地覆盖感知采样,丰富空间与语义覆盖。通过将传感模态视为自监督方法中的自然增强,我们以模态特定的补丁嵌入和自适应交叉注意力融合统一雷达与光学输入。训练策略整合局部-全局对比学习,并引入双中心机制,结合类别频率感知正则化以缓解土地覆盖的长尾分布问题。TerraFM在分类与分割任务上均表现出强泛化能力,在GEO-Bench与Copernicus-Bench上优于先前模型。代码与预训练模型已公开:https://github.com/mbzuai-oryx/TerraFM。
原文摘要 · Abstract (English)
Modern Earth observation (EO) increasingly leverages deep learning to harness the scale and diversity of satellite imagery across sensors and regions. While recent foundation models have demonstrated promising generalization across EO tasks, many remain limited by the scale, geographical coverage, and spectral diversity of their training data, factors critical for learning globally transferable representations. In this work, we introduce TerraFM, a scalable self-supervised learning model that leverages globally distributed Sentinel-1 and Sentinel-2 imagery, combined with large spatial tiles and land-cover aware sampling to enrich spatial and semantic coverage. By treating sensing modalities as natural augmentations in our self-supervised approach, we unify radar and optical inputs via modality-specific patch embeddings and adaptive cross-attention fusion. Our training strategy integrates local-global contrastive learning and introduces a dual-centering mechanism that incorporates class-frequency-aware regularization to address long-tailed distributions in land cover.TerraFM achieves strong generalization on both classification and segmentation tasks, outperforming prior models on GEO-Bench and Copernicus-Bench. Our code and pretrained models are publicly available at: https://github.com/mbzuai-oryx/TerraFM .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。