提出高效多尺度变压器，提升长期天气预报精度与速度。
EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecasting
- 用单次卷积提取多尺度特征，降低计算开销。
- 采用累积上下文微调，提升长时间序列一致性。
- 适合需要高精度长期预测的气象与视觉任务。
长期天气预报对社会经济规划和灾害预防至关重要。尽管近期方法通过微调扩展预测范围，但仍受限于灾难性遗忘、误差积累和高训练开销。为此，我们提出一个涵盖预训练、微调和预报的新流程，以增强长上下文建模并降低计算负担。首先，引入高效多尺度变换器（EMFormer），在训练和推理中均通过一次卷积提取多尺度特征。基于此架构，进一步采用累积上下文微调，在不损害短期精度的前提下提升时间一致性。此外，提出复合损失函数，通过正弦加权动态平衡各损失项，自适应引导预训练与微调阶段的优化轨迹。实验表明，该方法在天气预报和极端事件预测中表现优异，显著提升长期预测准确率。同时，EMFormer在视觉基准（ImageNet-1K 和 ADE20K）上展现强泛化能力，相比传统多尺度模块提速 5.69 倍。
原文摘要 · Abstract (English)
Long-term weather forecasting is critical for socioeconomic planning and disaster preparedness. While recent approaches employ finetuning to extend prediction horizons, they remain constrained by the issues of catastrophic forgetting, error accumulation, and high training overhead. To address these limitations, we present a novel pipeline across pretraining, finetuning and forecasting to enhance long-context modeling while reducing computational overhead. First, we introduce an Efficient Multi-scale Transformer (EMFormer) to extract multi-scale features through a single convolution in both training and inference. Based on the new architecture, we further employ an accumulative context finetuning to improve temporal consistency without degrading short-term accuracy. Additionally, we propose a composite loss that dynamically balances different terms via a sinusoidal weighting, thereby adaptively guiding the optimization trajectory throughout pretraining and finetuning. Experiments show that our approach achieves strong performance in weather forecasting and extreme event prediction, substantially improving long-term forecast accuracy. Moreover, EMFormer demonstrates strong generalization on vision benchmarks (ImageNet-1K and ADE20K) while delivering a 5.69x speedup over conventional multi-scale modules. Code: https://github.com/chenhao-zju/emformer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。