用物理约束微调,让DeepONet跨算子零样本泛化
DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuning
- 分布式预训练融合多类函数数据,奠定通用基础
- 零样本微调结合物理损失,新任务仅需少量数据即达高精度
- 适合需要快速适配新偏微分方程问题的研究者
我们提出一种新型微调方法,通过分布式神经算子预训练整合多样函数数据,再以物理信息损失实现零样本微调,完成多算子学习。神经算子能有效近似偏微分方程(PDE)及其相关问题的解算子,但常难以泛化到新任务。为此,我们研究预训练模型的微调策略,精心选择初始化以实现快速适应,仅用极少下游数据即可完成任务迁移。该方法在预训练阶段采用分布式学习融合多类算子数据,微调阶段利用物理信息损失实现零样本适配,降低对下游数据的依赖。我们对比了标准微调与低秩适应(LoRA)微调,应用于复杂非线性目标算子的学习,这些算子仅靠随机初始化难以训练。通过全面数值实验,验证了该方法在精度上的显著提升。研究成果为多算子学习提供了稳健框架,并凸显了迁移学习在该领域的潜力。
原文摘要 · Abstract (English)
We propose a novel fine-tuning method to achieve multi-operator learning through training a distributed neural operator with diverse function data and then zero-shot fine-tuning the neural network using physics-informed losses for downstream tasks. Operator learning effectively approximates solution operators for PDEs and various PDE-related problems, yet it often struggles to generalize to new tasks. To address this, we investigate fine-tuning a pretrained model, while carefully selecting an initialization that enables rapid adaptation to new tasks with minimal data. Our approach combines distributed learning to integrate data from various operators in pre-training, while physics-informed methods enable zero-shot fine-tuning, minimizing the reliance on downstream data. We investigate standard fine-tuning and Low-Rank Adaptation fine-tuning, applying both to train complex nonlinear target operators that are difficult to learn only using random initialization. Through comprehensive numerical examples, we demonstrate the advantages of our approach, showcasing significant improvements in accuracy. Our findings provide a robust framework for advancing multi-operator learning and highlight the potential of transfer learning techniques in this domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。