用时空分解的Transformer模型，让AI更高效预测全球天气。
WeatherFormer: Empowering Global Numerical Weather Forecasting with Space-Time Transformer
- 设计时空分离的Transformer结构，减少参数和内存占用。
- 在WeatherBench数据集上超越现有深度学习方法，接近物理模型性能。
- 适合关注气候智能预测与低碳计算的研究者与开发者。
数值天气预报（NWP）系统对现代社会影响深远。传统NWP依赖复杂偏微分方程求解，需大型计算集群，导致高碳排放。探索高效环保的NWP解决方案成为人工智能与地球科学界的共同关注。为缩小基于AI的方法与物理模型之间的性能差距，本文提出一种新的基于Transformer的NWP框架——WeatherFormer，用于建模复杂的时空大气动态，提升数据驱动型NWP能力。WeatherFormer创新性地引入时空因子化Transformer模块，显著降低参数量与内存消耗；其中提出的定位感知自适应傅里叶神经算子（PAFNO）实现位置敏感的特征混合。此外，采用两种数据增强策略以提升性能并降低训练开销。在WeatherBench数据集上的大量实验表明，WeatherFormer在多个评估指标上优于现有深度学习方法，并进一步逼近最先进物理模型的表现。
原文摘要 · Abstract (English)
Numerical Weather Prediction (NWP) system is an infrastructure that exerts considerable impacts on modern society.Traditional NWP system, however, resolves it by solving complex partial differential equations with a huge computing cluster, resulting in tons of carbon emission. Exploring efficient and eco-friendly solutions for NWP attracts interest from Artificial Intelligence (AI) and earth science communities. To narrow the performance gap between the AI-based methods and physic predictor, this work proposes a new transformer-based NWP framework, termed as WeatherFormer, to model the complex spatio-temporal atmosphere dynamics and empowering the capability of data-driven NWP. WeatherFormer innovatively introduces the space-time factorized transformer blocks to decrease the parameters and memory consumption, in which Position-aware Adaptive Fourier Neural Operator (PAFNO) is proposed for location sensible token mixing. Besides, two data augmentation strategies are utilized to boost the performance and decrease training consumption. Extensive experiments on WeatherBench dataset show WeatherFormer achieves superior performance over existing deep learning methods and further approaches the most advanced physical model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。