一个模型搞定多分辨率、多尺度、多模态遥感数据
AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities
- 用联合嵌入预测架构+自适应空间编码器统一处理异构遥感数据
- 在5个数据集11种传感器上训练，6个外部任务达顶尖性能
- 适合需要跨数据源、多任务的环境监测研究者使用
地理空间模型需适应遥感数据在分辨率、尺度和模态上的多样性。现有方法通常要求固定输入配置，限制了实际应用。我们提出AnySat，一种基于联合嵌入预测架构（JEPA）和尺度自适应空间编码器的多模态模型，可对高度异构数据进行自监督训练。为验证该统一方法的优势，我们构建了GeoPlex，一个包含5个多模态数据集的集合，涵盖11种不同传感器。我们在这些多样化数据集上同时训练单一强大模型。微调或探查后，该模型在GeoPlex测试集及6个外部数据集上，在土地覆盖制图、树种识别、作物类型分类、变化检测、气候类型分类以及洪水、烧毁区和森林砍伐区域分割等任务中均达到当前最优表现。代码与模型已公开于https://github.com/gastruc/AnySat。
原文摘要 · Abstract (English)
Geospatial models must adapt to the diversity of Earth observation data in terms of resolutions, scales, and modalities. However, existing approaches expect fixed input configurations, which limits their practical applicability. We propose AnySat, a multimodal model based on joint embedding predictive architecture (JEPA) and scale-adaptive spatial encoders, allowing us to train a single model on highly heterogeneous data in a self-supervised manner. To demonstrate the advantages of this unified approach, we compile GeoPlex, a collection of 5 multimodal datasets with varying characteristics and $11$ distinct sensors. We then train a single powerful model on these diverse datasets simultaneously. Once fine-tuned or probed, we reach state-of-the-art results on the test sets of GeoPlex and for 6 external datasets across various environment monitoring tasks: land cover mapping, tree species identification, crop type classification, change detection, climate type classification, and segmentation of flood, burn scar, and deforestation. The code and models are available at https://github.com/gastruc/AnySat.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。