优化策略比网络结构更重要,能显著提升气候预测模型性能。
Maximizing the Impact of Deep Learning on Subseasonal-to-Seasonal Climate Forecasting: The Essential Role of Optimization
- 提出多阶段优化方法,解决训练中误差累积问题。
- 在相同结构下,性能超越顶尖气象预报系统19%-91%。
- 适合气候建模、深度学习应用者参考,尤其关注长周期预测。
天气与气候预测对农业、防灾等领域至关重要。尽管数值天气预报(NWP)系统不断进步,但在2至6周的次季节至季节尺度(S2S)上,由于大气信号混沌且稀疏,预测仍具挑战性。即使最先进的深度学习模型也难以超越简单气候学模型。本文发现,性能差距的根源在于优化而非网络结构,并提出一种新型多阶段优化策略以弥补这一差距。大量实证研究表明,该方法在关键技能指标PCC和TCC上均有显著提升,且使用相同骨干结构时,超越主流NWP系统(ECMWF-S2S)达19%-91%。研究还反驳了“直接预测优于滚动预测”的观点,通过理论分析指出滚动预测表现不佳可能源于训练中雅可比矩阵乘积的累积效应。所提多阶段框架可视为一种教师强制机制来缓解此问题。代码已公开于https://anonymous.4open.science/r/Baguan-S2S-23E7/。
原文摘要 · Abstract (English)
Weather and climate forecasting is vital for sectors such as agriculture and disaster management. Although numerical weather prediction (NWP) systems have advanced, forecasting at the subseasonal-to-seasonal (S2S) scale, spanning 2 to 6 weeks, remains challenging due to the chaotic and sparse atmospheric signals at this interval. Even state-of-the-art deep learning models struggle to outperform simple climatology models in this domain. This paper identifies that optimization, instead of network structure, could be the root cause of this performance gap, and then we develop a novel multi-stage optimization strategy to close the gap. Extensive empirical studies demonstrate that our multi-stage optimization approach significantly improves key skill metrics, PCC and TCC, while utilizing the same backbone structure, surpassing the state-of-the-art NWP systems (ECMWF-S2S) by over \textbf{19-91\%}. Our research contests the recent study that direct forecasting outperforms rolling forecasting for S2S tasks. Through theoretical analysis, we propose that the underperformance of rolling forecasting may arise from the accumulation of Jacobian matrix products during training. Our multi-stage framework can be viewed as a form of teacher forcing to address this issue. Code is available at \url{https://anonymous.4open.science/r/Baguan-S2S-23E7/}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。