用预训练缓解天气预报过拟合，模型表现更稳定准确
Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model
- 设计挑战性预训练任务，引入局部偏差抑制过拟合
- 在中期天气预报中优于传统方法，且适应不同预报时长
- 适合需要高鲁棒性的气候建模与区域预报场景
天气预报长期是人类面临的重大挑战。尽管近年基于AI的模型在全局预报任务上已超越传统数值天气预报（NWP）方法，但受限于真实气象数据仅覆盖数十年，过拟合仍是关键问题。与计算机视觉或自然语言处理等数据丰富的领域不同，天气预报需在现有数据下创新应对策略。本文研究天气预报的预训练方法，发现选择适当挑战性的预训练任务可引入局部偏差，有效缓解过拟合并提升性能。我们提出Baguan，一种基于孪生自编码器、自监督预训练并针对不同预报时效微调的数据驱动中期天气预报模型。实验表明，Baguan在中期预报中优于传统方法，且预训练后的模型在下游任务如次季节到季节（S2S）建模和区域预报中表现出强鲁棒性与泛化能力。
原文摘要 · Abstract (English)
Weather forecasting has long posed a significant challenge for humanity. While recent AI-based models have surpassed traditional numerical weather prediction (NWP) methods in global forecasting tasks, overfitting remains a critical issue due to the limited availability of real-world weather data spanning only a few decades. Unlike fields like computer vision or natural language processing, where data abundance can mitigate overfitting, weather forecasting demands innovative strategies to address this challenge with existing data. In this paper, we explore pre-training methods for weather forecasting, finding that selecting an appropriately challenging pre-training task introduces locality bias, effectively mitigating overfitting and enhancing performance. We introduce Baguan, a novel data-driven model for medium-range weather forecasting, built on a Siamese Autoencoder pre-trained in a self-supervised manner and fine-tuned for different lead times. Experimental results show that Baguan outperforms traditional methods, delivering more accurate forecasts. Additionally, the pre-trained Baguan demonstrates robust overfitting control and excels in downstream tasks, such as subseasonal-to-seasonal (S2S) modeling and regional forecasting, after fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。