用超大模型实现全球高分辨率气象预报，突破传统方法局限。
ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling
- 采用轻量残差架构与分块序列算法，解决注意力计算瓶颈。
- 支持0.9公里全球分辨率，7公里精度达0.98-0.99相关系数。
- 适合气候研究、灾害预警等需要高精度区域预测的场景。
稀疏观测和粗分辨率气候模型限制了区域决策的有效性，亟需稳健的降尺度方法。现有AI方法在变量与地理泛化上表现不佳，且受限于视觉变换器（ViT）自注意力的二次复杂度。我们提出ORBIT-2，一种面向全球超分辨率气候降尺度的可扩展基础模型。其包含两项关键创新：(1) 残差瘦身视觉变换器（Reslim），通过残差学习与贝叶斯正则化实现高效稳健预测；(2) 分块序列缩放算法（TILES），将自注意力复杂度从二次降至线性，支持长序列处理与大规模并行。ORBIT-2 在65,536个GPU上扩展至100亿参数，实现高达4.1 exaFLOPS持续吞吐量，强缩放效率达74–98%。支持0.9公里全球分辨率降尺度，处理最长达42亿标记的序列。在7公里分辨率基准测试中，对观测数据的 $R^2$ 得分达到0.98–0.99，表现优异。
原文摘要 · Abstract (English)
Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, hyper-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 65,536 GPUs, achieving up to 4.1 exaFLOPS sustained throughput and 74--98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. On 7 km resolution benchmarks, ORBIT-2 achieves high accuracy with $R^2$ scores in the range of 0.98--0.99 against observational data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。