让神经网络在数据分布不同时仍保持性能,通过权重异常修正实现跨域泛化。
Conditional updates of neural network weights for increased out of training performance
- 从训练数据子集重训并记录权重异常,构建预测模型。
- 用预测变量回归权重异常,实现对新数据的权重外推。
- 在气候科学三类场景中成功实现时空及跨领域迁移。
本研究提出一种提升神经网络在训练数据与应用数据差异较大(如分布外问题或模式/制度变化)时性能的方法。该方法包含三个步骤:1)在合理子集上重新训练神经网络,记录权重异常;2)选择合理预测变量,建立其与权重异常间的回归关系;3)外推权重以适应应用数据。本文在气候科学三个应用场景中验证了该方法,成功实现了神经网络在时间、空间和跨领域上的外推。
原文摘要 · Abstract (English)
This study proposes a method to enhance neural network performance when training data and application data are not very similar, e.g., out of distribution problems, as well as pattern and regime shifts. The method consists of three main steps: 1) Retrain the neural network towards reasonable subsets of the training data set and note down the resulting weight anomalies. 2) Choose reasonable predictors and derive a regression between the predictors and the weight anomalies. 3) Extrapolate the weights, and thereby the neural network, to the application data. We show and discuss this method in three use cases from the climate sciences, which include successful temporal, spatial and cross-domain extrapolations of neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。