用极少量高质量数据训练小型高效模型，快速求解参数化偏微分方程。
S$^2$GPT-PINNs: Sparse and Small models for PDEs
- 通过任务特异性激活函数迁移和物理损失点降采样实现双层定制化压缩
- 参数量仅为传统PINNs的数个数量级，仍保持高精度求解能力
- 适合资源受限场景下的科学计算与实时仿真，如工程优化与逆问题
我们提出 S²GPT-PINN，一种用于求解参数化偏微分方程（PDEs）的小型稀疏模型。该模型类似小型语言模型（SLMs），专为特定类型的PDE家族设计，具有紧凑架构和极低计算开销。利用由大尺度全阶模型支持的数学严谨贪心算法，仅需极少量高质量数据即可训练。S²GPT-PINN 通过两层定制化机制，在参数量远少于传统PINNs的情况下实现极高效率：第一层是通过任务特异性激活函数从预训练PINNs中进行知识迁移；第二层是在计算物理信息损失时实施精心设计的降采样，将数据点数量压缩至小模型规模的数个数量级。
原文摘要 · Abstract (English)
We propose S$^2$GPT-PINN, a sparse and small model for solving parametric partial differential equations (PDEs). Similar to Small Language Models (SLMs), S$^2$GPT-PINN is tailored to domain-specific (families of) PDEs and characterized by its compact architecture and minimal computational power. Leveraging a small amount of extremely high quality data via a mathematically rigorous greedy algorithm that is enabled by the large full-order models, S$^2$GPT-PINN relies on orders of magnitude less parameters than PINNs to achieve extremely high efficiency via two levels of customizations. The first is knowledge distillation via task-specific activation functions that are transferred from Pre-Trained PINNs. The second is a judicious down-sampling when calculating the physics-informed loss of the network compressing the number of data sites by orders of magnitude to the size of the small model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。