无需真实解即可适配微分方程模型，提升泛化能力。
Unsupervised Adaptation of PDE Foundation Models
- 用物理残差和边界条件构建无监督目标函数，替代真实解。
- 在多维异构方程上性能媲美有监督微调，且优于主流神经算子方法。
- 提出NSLoRA优化适配均衡性，适合缺乏标注数据的科学计算场景。
预训练的偏微分方程（PDE）基础模型可在不同方程间实现泛化，但将其适配到未见的PDE系统通常需密集解数据，而这类数据往往昂贵或不可得。为解决此问题，我们提出一种基于PDE的无监督微调框架，无需真实解即可完成适配。首先，在涵盖不同空间尺度的时间依赖型PDE上预训练一个邻域注意力Transformer，获得跨异构方程可迁移的表示。在适配阶段，利用PDE残差与边界条件构建物理驱动的目标函数，并通过低秩适配（LoRA）对新方程进行微调。针对标准LoRA中物理量学习不均的问题，提出NSLoRA——一种牛顿-舒尔茨正交化变体，实现适配平衡。该方法在无需任何真实解的情况下，性能媲美监督式LoRA微调，且在跨越多个空间维度的异构PDE基准测试中持续优于竞争性神经算子基线与近期的PDE基础模型。
原文摘要 · Abstract (English)
Pretrained partial differential equation (PDE) foundation models can generalize across different equations, but adapting them to unseen PDE systems typically requires dense solution data, which is often expensive or unavailable. To address this limitation, we propose an unsupervised PDE-based finetuning framework that eliminates the need for ground-truth solutions. We first pretrain a neighborhood attention Transformer on diverse time-dependent PDEs spanning varying spatial scales, yielding transferable representations across heterogeneous equations. In the adaptation stage, we construct a physics-based objective using the PDE residual and boundary conditions, and finetune the model on unseen equations via low-rank adaptation (LoRA). To address the uneven learning across physical quantities in standard LoRA, we introduce NSLoRA, a Newton-Schulz orthogonalized variant that rebalances adaptation. Our method achieves performance comparable to supervised LoRA finetuning without requiring any ground-truth solutions, while consistently outperforming competitive neural operator baselines and recent PDE foundation models across heterogeneous PDE benchmarks spanning multiple spatial dimensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。