用轻量U-Net模型高效求解各类偏微分方程，参数少、泛化强。
SPUS: A Lightweight and Parameter-Efficient Foundation Model for PDEs
- 基于残差U-Net设计，架构简洁，适合作为偏微分方程通用求解器。
- 在6个未见过的下游任务中实现当前最优泛化性能，仅需少量微调数据。
- 通过自回归预训练学习物理规律，适合需要低资源部署的科研与工程场景。
我们提出小型偏微分方程U-Net求解器（SPUS），一种紧凑高效的偏微分方程基础模型（FM），作为统一神经算子用于求解多种偏微分方程。不同于现有主流方法依赖高复杂度的Transformer架构（带来高计算与参数开销），SPUS采用较少被探索的轻量级残差U-Net结构。为在该极简框架下实现有效学习，我们设计了一种简单但强大的自回归预训练策略，紧密模拟数值求解器行为以捕捉底层物理规律。SPUS在多样化流体动力学偏微分方程上进行预训练，并在6个具有挑战性的未见下游任务上评估，涵盖多种物理系统。实验表明，使用残差U-Net结构的SPUS在这些下游任务中实现了当前最优的泛化性能，同时所需参数显著减少且仅需极少微调数据，展现出其作为高参数效率偏微分方程基础模型的巨大潜力。
原文摘要 · Abstract (English)
We introduce Small PDE U-Net Solver (SPUS), a compact and efficient foundation model (FM) designed as a unified neural operator for solving a wide range of partial differential equations (PDEs). Unlike existing state-of-the-art PDE FMs-primarily based on large complex transformer architectures with high computational and parameter overhead-SPUS leverages a lightweight residual U-Net-based architecture that has been largely underexplored as a foundation model architecture in this domain. To enable effective learning in this minimalist framework, we utilize a simple yet powerful auto-regressive pretraining strategy which closely replicates the behavior of numerical solvers to learn the underlying physics. SPUS is pretrained on a diverse set of fluid dynamics PDEs and evaluated across 6 challenging unseen downstream PDEs spanning various physical systems. Experimental results demonstrate that SPUS using residual U-Net based architecture achieves state-of-the-art generalization on these downstream tasks while requiring significantly fewer parameters and minimal fine-tuning data, highlighting its potential as a highly parameter-efficient FM for solving diverse PDE systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。