用傅里叶分析揭示迁移学习如何提升海洋湍流亚格子模型泛化能力
Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence
- 通过傅里叶分析发现神经网络自动学习低通、伽伯和高通滤波器
- 无迁移学习时输出谱被低估，迁移学习仅微调一层即可纠正
- 适用于各类数据驱动的动态系统参数化，尤其适合小样本场景
迁移学习（TL）是提升天气与气候预测及湍流建模中神经网络性能的强大工具，可在新系统仅需少量数据时实现对分布外数据的泛化。本研究采用9层卷积神经网络（NN）预测双层海洋准地转系统中的亚格子强迫，并考察何种指标最能描述其性能与泛化能力。对神经网络核的傅里叶分析表明，无论训练数据是否各向同性，网络均学习到低通、伽伯和高通滤波器。通过对激活谱的分析，揭示了神经网络在无迁移学习时无法泛化的原因：从一个数据集学习的权重和偏置会低估分布外样本的谱，导致输出谱被低估。通过仅用目标系统数据微调一层，可修正该低估问题，使神经网络输出匹配目标谱。这些发现对数据驱动的动力系统参数化具有广泛适用性。
原文摘要 · Abstract (English)
Transfer learning (TL) is a powerful tool for enhancing the performance of neural networks (NNs) in applications such as weather and climate prediction and turbulence modeling. TL enables models to generalize to out-of-distribution data with minimal training data from the new system. In this study, we employ a 9-layer convolutional NN to predict the subgrid forcing in a two-layer ocean quasi-geostrophic system and examine which metrics best describe its performance and generalizability to unseen dynamical regimes. Fourier analysis of the NN kernels reveals that they learn low-pass, Gabor, and high-pass filters, regardless of whether the training data are isotropic or anisotropic. By analyzing the activation spectra, we identify why NNs fail to generalize without TL and how TL can overcome these limitations: the learned weights and biases from one dataset underestimate the out-of-distribution sample spectra as they pass through the network, leading to an underestimation of output spectra. By re-training only one layer with data from the target system, this underestimation is corrected, enabling the NN to produce predictions that match the target spectra. These findings are broadly applicable to data-driven parameterization of dynamical systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。