用物理规律微调科学大模型,数据少也能准
Physics-informed fine-tuning of foundation models for partial differential equations
- 在微调时直接加入微分方程和边界条件约束
- 仅用少量数据就达到与全量数据相当的精度
- 适合小样本、跨领域物理问题建模场景
偏微分方程(PDE)的基础模型作为预训练的强代理,在多样化物理系统上表现优异,但面对新下游任务时,因任务特定数据有限且分布漂移,适应困难。尽管在自然语言处理中微调已证明有效,但针对PDE基础模型的最佳实践仍不明确。虽然物理信息训练可广泛构建高精度求解器,但其在数据驱动基础模型微调中的潜力尚未系统研究。本文提出一种物理信息微调框架,通过将PDE残差和边界条件直接融入微调目标,实现对预训练模型的有效适配,尤其在数据稀缺情况下仍保持物理一致性。我们在一个未见过的PDE类别任务上评估该方法,结果表明:无需真实解即可达到竞争性精度;当仅有极少训练数据时,混合微调策略展现出更优的分布外泛化能力。这些发现确立了物理信息微调作为一种可扩展、数据高效的科学机器学习模型适配范式。
原文摘要 · Abstract (English)
Foundation models for partial differential equations (PDEs) have emerged as powerful surrogates pre-trained on diverse physical systems, but adapting them to new downstream tasks remains challenging due to limited task-specific data and distribution shifts. While fine-tuning has proven transformative in natural language processing, best practices for adapting PDE foundation models remain underexplored. Although physics-informed training has successfully trained accurate solvers across a wide range of PDE problems, its potential for fine-tuning data-based foundation models has not been systematically studied. In this work, we introduce a physics-informed fine-tuning framework that adapts pre-trained PDE foundation models by incorporating physical constraints (PDE residuals and boundary conditions) directly into the fine-tuning objective. This enables effective adaptation in data-scarce regimes while promoting physical consistency. We evaluate our method on a downstream task composed of an unseen PDE class and compare it with data-driven finetuning counterparts. Our results demonstrate that physics-informed fine-tuning achieves competitive accuracy without requiring PDE solutions for training. Furthermore, a hybrid fine-tuning strategy yields superior generalization to out-of-distribution scenarios when only minimal training data is available. These findings establish physics-informed fine-tuning as a scalable and data-efficient paradigm, providing a physically interpretable pathway for adapting foundation models in scientific machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。