用卷积神经算子实现微分方程的少样本迁移学习,精度显著提升。
Convolutional-neural-operator-based transfer learning for solving PDEs
- 基于预训练卷积神经算子,通过小样本微调参数解决微分方程
- 在柯马托-西瓦辛斯基等方程上,新方法比基线模型更准确
- 神经元线性变换策略效果最佳,适合快速适应新问题
卷积神经算子是一种基于CNN的架构,能保持结构不变性,实现微分方程解算子的真实、无混叠学习。该模型在某些情况下已优于DeepONet、傅里叶神经算子和Galerkin Transformer。然而,其在少样本学习中的表现尚未验证。本文通过先在源数据集上预训练卷积神经算子,再仅用少量目标数据调整参数,将其拓展至少样本场景。研究了三种参数调整策略:微调、低秩适配和神经元线性变换,结果表明神经元线性变换在求解柯马托-西瓦辛斯基方程、布鲁塞尔扩散-反应系统及纳维-斯托克斯方程时取得最高近似精度。
原文摘要 · Abstract (English)
Convolutional neural operator is a CNN-based architecture recently proposed to enforce structure-preserving continuous-discrete equivalence and enable the genuine, alias-free learning of solution operators of PDEs. This neural operator was demonstrated to outperform for certain cases some baseline models such as DeepONet, Fourier neural operator, and Galerkin transformer in terms of surrogate accuracy. The convolutional neural operator, however, seems not to be validated for few-shot learning. We extend the model to few-shot learning scenarios by first pre-training a convolutional neural operator using a source dataset and then adjusting the parameters of the trained neural operator using only a small target dataset. We investigate three strategies for adjusting the parameters of a trained neural operator, including fine-tuning, low-rank adaption, and neuron linear transformation, and find that the neuron linear transformation strategy enjoys the highest surrogate accuracy in solving PDEs such as Kuramoto-Sivashinsky equation, Brusselator diffusion-reaction system, and Navier-Stokes equations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。