提出SLE-FNO模型,实现无需旧数据的持续学习,避免遗忘且参数增长极少。
SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operators
- 在FNO基础上添加单层扩展,通过轻量结构支持持续学习
- 在230组流体模拟任务中零遗忘,性能优于主流方法
- 适合需要外推、无法访问历史数据的科学建模场景
科学机器学习广泛用于构建代理模型,但多数模型假设未来数据分布与训练集一致。实际中,新实验条件或模拟工况可能显著不同,需在不重访历史数据的情况下进行外推和模型更新。这催生了需适应分布偏移且防止灾难性遗忘的持续学习(CL)框架。此类挑战在流体动力学中尤为突出,几何、边界条件或流动模式变化会带来解的显著改变。本文提出一种基于架构的新方法SLE-FNO,将单层扩展(SLE)与傅里叶神经算子(FNO)结合,实现高效持续学习。在空间场到场回归任务中,对比了包括EWC、LwF、基于回放的方法、OGD、GEM、PiggyBack和LoRA在内的多种主流CL方法。模型以230组计算流体动力学模拟为基础,分四阶段、跨分布的任务序列训练,目标是将瞬态浓度场映射为脉动动脉瘤血流中的时间平均壁面剪切应力(TAWSS)。结果表明,基于回放和架构的方法(PiggyBack、LoRA、SLE-FNO)表现最佳,其中SLE-FNO在可塑性与稳定性间取得最优平衡,实现零遗忘,仅增加少量参数。研究揭示了不同CL算法的关键差异,并提出SLE-FNO是一种在需外推时适应基础模型的有前景策略。
原文摘要 · Abstract (English)
Scientific machine learning is increasingly used to build surrogate models, yet most models are trained under a restrictive assumption in which future data follow the same distribution as the training set. In practice, new experimental conditions or simulation regimes may differ significantly, requiring extrapolation and model updates without re-access to prior data. This creates a need for continual learning (CL) frameworks that can adapt to distribution shifts while preventing catastrophic forgetting. Such challenges are pronounced in fluid dynamics, where changes in geometry, boundary conditions, or flow regimes induce non-trivial changes to the solution. Here, we introduce a new architecture-based approach (SLE-FNO) combining a Single-Layer Extension (SLE) with the Fourier Neural Operator (FNO) to support efficient CL. SLE-FNO was compared with a range of established CL methods, including Elastic Weight Consolidation (EWC), Learning without Forgetting (LwF), replay-based approaches, Orthogonal Gradient Descent (OGD), Gradient Episodic Memory (GEM), PiggyBack, and Low-Rank Adaptation (LoRA), within a spatial field-to-field regression setting. The models were trained to map transient concentration fields to time-averaged wall shear stress (TAWSS) in pulsatile aneurysmal blood flow. Tasks were derived from 230 computational fluid dynamics simulations grouped into four sequential and out-of-distribution configurations. Results show that replay-based methods and architecture-based approaches (PiggyBack, LoRA, and SLE-FNO) achieve the best retention, with SLE-FNO providing the strongest overall balance between plasticity and stability, achieving accuracy with zero forgetting and minimal additional parameters. Our findings highlight key differences between CL algorithms and introduce SLE-FNO as a promising strategy for adapting baseline models when extrapolation is required.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。