23亿参数的气象气候基础模型,可统一处理预报、降尺度等多任务。
Prithvi WxC: Foundation Model for Weather and Climate

- 基于160变量的MERRA-2数据,用编码器-解码器结构捕捉全球与区域依赖关系。
- 在自回归预报、极端事件估计等4项任务中表现优异,支持高分辨率建模。
- 开源发布,适合气象研究者和气候建模开发者快速微调应用。
受人工智能模拟器在高性能计算系统上媲美传统数值天气预报模型的启发,越来越多大型AI模型被用于天气预报、降尺度或实时预报等场景。尽管人工智能领域聚焦于可适配多种任务的基础模型,但气象气候领域仍以单一任务为主,尤其侧重中期预报。本文填补这一空白,提出Prithvi WxC,一个使用160个变量的现代回顾性分析研究与应用第二版(MERRA-2)数据训练的23亿参数基础模型。该模型采用基于编码器-解码器的架构,融合近期多种Transformer设计思想,有效捕捉输入数据中的区域与全局依赖。模型支持大令牌数,可实现不同拓扑下高分辨率天气现象建模。训练采用混合目标,结合掩码重建与预报任务。我们在多个挑战性下游任务上测试:自回归滚动预报、降尺度、重力波通量参数化、极端事件估计。预训练模型及配套微调流程已通过Hugging Face开源发布。
原文摘要 · Abstract (English)
Triggered by the realization that AI emulators can rival the performance of traditional numerical weather prediction models running on HPC systems, there is now an increasing number of large AI models that address use cases such as forecasting, downscaling, or nowcasting. While the parallel developments in the AI literature focus on foundation models -- models that can be effectively tuned to address multiple, different use cases -- the developments on the weather and climate side largely focus on single-use cases with particular emphasis on mid-range forecasting. We close this gap by introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture, incorporating concepts from various recent transformer models to effectively capture both regional and global dependencies in the input data. The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. Furthermore, it is trained with a mixed objective that combines the paradigms of masked reconstruction with forecasting. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation. The pretrained model with 2.3 billion parameters, along with the associated fine-tuning workflows, has been publicly released as an open-source contribution via Hugging Face.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。