自动优化物理信息神经网络，提升微分方程求解效率与精度。
BO-SA-PINNs: Self-adaptive physics-informed neural networks based on Bayesian optimization for automatically designing PDE solvers
- 用贝叶斯优化自动搜索最优超参数和采样策略。
- 三阶段设计：先优化超参，再自适应调整损失权重与采样分布，最后用L-BFGS稳定训练。
- 在多个方程上验证有效，适合希望减少人工调参的科研人员。
物理信息神经网络（PINNs）正成为求解偏微分方程（PDEs）的一种流行方法，但其对不同方程需手动调参，影响求解效率。本文提出通用多阶段框架BO-SA-PINNs以缓解此问题。第一阶段采用贝叶斯优化（BO）自动选择训练超参数，基于预训练结果确定网络结构、学习率、采样点分布及损失权重。所提基于实验结果的超参数搜索空间提升了BO效率。第二阶段引入全局自适应（SA）机制：利用预训练模型与损失信息，通过指数移动平均（EMA）优化损失权重，结合残差驱动的自适应分布优化（RAR-D）改进采样点分布。第三阶段使用L-BFGS实现稳定训练，并引入新激活函数以提升精度。在Helmholtz、Maxwell、Burgers及高维泊松方程上的对比与消融实验证明，该模型在测试案例中具有更高准确率和更少迭代次数，各改进均具正向贡献。
原文摘要 · Abstract (English)
Physics-informed neural networks (PINNs) is becoming a popular alternative method for solving partial differential equations (PDEs). However, they require dedicated manual modifications to the hyperparameters of the network, the sampling methods and loss function weights for different PDEs, which reduces the efficiency of the solvers. In this paper, we pro- pose a general multi-stage framework, i.e. BO-SA-PINNs to alleviate this issue. In the first stage, Bayesian optimization (BO) is used to select hyperparameters for the training process, and based on the results of the pre-training, the network architecture, learning rate, sampling points distribution and loss function weights suitable for the PDEs are automatically determined. The proposed hyperparameters search space based on experimental results can enhance the efficiency of BO in identifying optimal hyperparameters. After selecting the appropriate hyperparameters, we incorporate a global self-adaptive (SA) mechanism the second stage. Using the pre-trained model and loss information in the second-stage training, the exponential moving average (EMA) method is employed to optimize the loss function weights, and residual-based adaptive refinement with distribution (RAR-D) is used to optimize the sampling points distribution. In the third stage, L-BFGS is used for stable training. In addition, we introduce a new activation function that enables BO-SA-PINNs to achieve higher accuracy. In numerical experiments, we conduct comparative and ablation experiments to verify the performance of the model on Helmholtz, Maxwell, Burgers and high-dimensional Poisson equations. The comparative experiment results show that our model can achieve higher accuracy and fewer iterations in test cases, and the ablation experiments demonstrate the positive impact of every improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。