预生成多难度微分方程数据,显著减少高难度问题训练所需样本量。
Pre-Generating Multi-Difficulty PDE Data for Few-Shot Neural PDE Solvers
- 通过预生成低中难度问题数据,提升高难度问题学习效率。
- 仅用8.9倍更少的计算量即可达到与全高难度数据相当的精度。
- 适合需要高效训练神经微分方程求解器的研究者参考。
学习型偏微分方程(PDE)求解器的主要成本常来自经典求解器生成训练数据,而非模型学习本身。同时,问题难度存在明确维度——如复杂几何结构和更高雷诺数——这些使经典求解更困难,也更可能从神经加速中受益。为解决这一循环难题,我们系统研究二维不可压缩纳维-斯托克斯方程中沿几何(障碍物数量与位置)、物理(雷诺数)及其组合的难度迁移。类似于预训练基础模型以提升下游任务表现,我们发现:预先用经典方法求解大量低、中难度样本并纳入训练集,可显著减少学习高难度物理所需的样本数。此外,通过结合低、高难度数据,仅需8.9倍更少的预生成计算量,即可达到仅使用高难度数据时的误差水平。结果表明,分配经典求解器计算资源于不同难度层级的方式,与总计算量同等重要,提示对预生成PDE数据进行有策略的筛选能带来显著收益。代码已开源。
原文摘要 · Abstract (English)
A key aspect of learned partial differential equation (PDE) solvers is that the main cost often comes from generating training data with classical solvers rather than learning the model itself. Another is that there are clear axes of difficulty--e.g., more complex geometries and higher Reynolds numbers--along which problems become (1) harder for classical solvers and thus (2) more likely to benefit from neural speedups. Towards addressing this chicken-and-egg challenge, we study difficulty transfer on 2D incompressible Navier-Stokes, systematically varying task complexity along geometry (number and placement of obstacles), physics (Reynolds number), and their combination. Similar to how it is possible to spend compute to pre-train foundation models and improve their performance on downstream tasks, we find that by classically solving (analogously pre-generating) many low and medium difficulty examples and including them in the training set, it is possible to learn high-difficulty physics from far fewer samples. Furthermore, we show that by combining low and high difficulty data, we can spend 8.9x less compute on pre-generating a dataset to achieve the same error as using only high difficulty examples. Our results highlight that how we allocate classical-solver compute across difficulty levels is as important as how much we allocate overall, and suggest substantial gains from principled curation of pre-generated PDE data for neural solvers. Our code is available at https://github.com/Naman-Choudhary-AI-ML/pregenerating-pde
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。