首个统一处理多源地球数据的通用模型,可预测无观测区域的气象变量。
Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting

- 用统一架构融合卫星、站点等异构数据,支持缺失值处理
- 通过轴向注意力捕捉变量间关系,实现无数据区的精准预测
- 适配极端天气预报,适合气候研究与灾害预警场景
地球系统基础模型(ESFM)基于先锋模型Aurora的3D Swin UNet主干网络,构建了首个完全开源的统一框架。该模型扩展编码方案与训练协议,可处理包含时空缺失值的多样化数据,如卫星遥感、地面站点数据。引入轴向注意力机制以捕捉变量间依赖关系,实现对初始时刻无观测区域(如特定气压层)的变量预测,同时保持温度、压力、湿度等变量间的物理一致性。通过变量独立标记,支持训练时变量随机组合,简化下游任务扩展。采用基于自适应层归一化的集成方法,将确定性模型轻松转为概率预测。实验涵盖密集网格数据(ERA5、CMIP6)、区域遮蔽数据、稀疏网格MODIS卫星数据及站点数据。结果表明性能优于或媲美现有最优基准。对2023年台风杜苏芮和2024年平流层突发变暖事件的案例研究,准确预测了极端天气的位置与强度。ESFM在保持长期稳定性的同时,显著拓展了在气候科学中的应用潜力。
原文摘要 · Abstract (English)
Foundation models (FMs) for the Earth system learn statistical relationships between physical variables across massive datasets to enable versatile downstream applications through finetuning, separating them from task-specific weather models. Here, we introduce Earth System Foundation Model (ESFM), a fully open model building on the 3D Swin UNet backbone of the pioneering Aurora model. ESFM introduces extensions that increase functionality and foster adoption in climate sciences. First, the encoding scheme and training protocols have been extended to handle diverse datasets, including those containing missing values across all spatio-temporal dimensions such as satellite data, as well as station data, all under one backbone. Axial attention is introduced to capture inter-variable dependencies. As a result ESFM skillfully predicts variables in regions or on pressure levels where no data is present at the initial time, while preserving inter-variable relationships, for example between temperature, pressure, and humidity. Individual variable tokenization enables different sets of variables to be shuffled during training and simplifies the process of building extensions for new downstream tasks. Adaptive layer norm-based ensembles allow for a simple yet effective way to transform deterministic ESFM to a probabilistic FM. We present findings using dense gridded data (ERA5, CMIP6), regionally masked dense data, sparse gridded MODIS satellite data, and station data. Results demonstrate competitive or superior performance relative to state-of-the-art benchmarks. Case studies of Super Typhoon Doksuri (2023) and 2024 sudden stratospheric warming events show accurate positional and magnitude estimations of extreme weather. ESFM retains the strengths of previous foundation models, such as long-term stability, but facilitates application to a variety of downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。