用进化算法分两阶段优化物理神经网络超参数,提升精度与稳定性。
Evolutionary Two-Stage Hyperparameter Optimization Strategies for Physics-Informed Neural Networks

- 先低精度快速筛选候选超参数配置,再精选配置全量训练。
- 在三个方程上平均误差显著降低,计算资源固定下表现更优。
- 适合对模型鲁棒性要求高、超参数调优困难的科学计算场景。
物理信息神经网络(PINNs)通过将物理定律嵌入神经网络训练来求解偏微分方程(PDE),但其性能易受架构与优化超参数影响,存在收敛不稳定、训练停滞及对超参数敏感等问题,根源在于物理约束损失函数高度非凸且多目标。在此背景下,外层超参数搜索是一个噪声大、黑箱化的异构参数优化问题,传统局部或基于梯度的方法易陷入次优区域。进化算法因其群体探索能力与对混合、不可导空间的适应性,更适合此类问题。本文提出并研究一种基于进化算法的两阶段方法,结合了训练中的探索与利用阶段,在固定计算预算下提升解的准确性和鲁棒性。第一阶段采用截断训练轮次进行低精度快速筛选,将超参数选择视为黑箱外层优化问题;第二阶段仅对最优秀候选配置使用标准梯度优化器完成完整训练以精炼解。在阿德维克、克莱因-戈登和亥姆霍兹方程三个典型问题上评估,本方法一致优于标准训练,显著降低平均误差。
原文摘要 · Abstract (English)
Physics-Informed Neural Networks (PINNs) solve Partial Differential Equations (PDEs) by embedding physical laws into neural network training. However, their performance suffers from unstable convergence, training plateaus, and strong sensitivity to architectural and optimization hyperparameters due to the highly non-convex and multi-term structure of the physics-informed loss. In this setting, the outer-loop hyperparameter search is a noisy and black-box optimization problem over heterogeneous parameters, where classical local or gradient-based strategies are easily trapped in suboptimal regions. Evolutionary algorithms, with their population-based exploration and ability to handle mixed, non-differentiable search spaces, provide a more robust mechanism for discovering promising configurations. We propose and investigate a two-stage approach based on evolutionary algorithms that combines exploration and exploitation parts of PINNs training to improve solution accuracy and robustness under fixed computational budgets. In the first stage, we perform low-fidelity training runs with truncated epochs to rapidly screen candidate configurations, treating hyperparameter selection as a black-box outer-loop problem. In the second stage, only the most promising candidates are fully trained with standard gradient-based optimizers to refine the solution. Evaluated on three popular problems, namely Advection, Klein-Gordon and Helmholtz equations, our method consistently outperforms standard training and achieves significantly lower mean error within constrained computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。