针对物理神经网络训练中的梯度失衡问题，提出自适应优化器提升求解稳定性。
PDE-aware Optimizer for Physics-informed Neural Networks
- 根据样本级残差梯度方差动态调整参数更新，缓解梯度方向冲突。
- 在1D伯格斯、艾伦-卡恩和科特韦格-德弗里斯方程上误差更低，收敛更平滑。
- 无需二阶计算开销，适合复杂偏微分方程的稳定求解，适合科研人员参考。
物理信息神经网络（PINNs）通过将物理约束嵌入损失函数，成为求解偏微分方程（PDEs）的强大框架。然而，标准优化器如Adam在刚性或病态系统中常难以平衡竞争性损失项。本文提出一种PDE感知优化器，基于每样本PDE残差梯度的方差自适应调整参数更新，无需高昂的二阶优化器计算成本（如SOAP）。在1维伯格斯（Burgers）、艾伦-卡恩（Allen-Cahn）和科特韦格-德弗里斯（KdV）方程上对比Adam与SOAP，PDE感知优化器实现更平滑的收敛和更低的绝对误差，尤其在梯度陡峭区域表现更优。结果表明，基于PDE残差的自适应机制能有效提升PINNs训练的稳定性。未来研究需进一步探索其在更大模型和硬件加速器上的扩展性。
原文摘要 · Abstract (English)
Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical constraints into the loss function. However, standard optimizers such as Adam often struggle to balance competing loss terms, particularly in stiff or ill-conditioned systems. In this work, we propose a PDE-aware optimizer that adapts parameter updates based on the variance of per-sample PDE residual gradients. This method addresses gradient misalignment without incurring the heavy computational costs of second-order optimizers such as SOAP. We benchmark the PDE-aware optimizer against Adam and SOAP on 1D Burgers', Allen-Cahn and Korteweg-de Vries(KdV) equations. Across both PDEs, the PDE-aware optimizer achieves smoother convergence and lower absolute errors, particularly in regions with sharp gradients. Our results demonstrate the effectiveness of PDE residual-aware adaptivity in enhancing stability in PINNs training. While promising, further scaling on larger architectures and hardware accelerators remains an important direction for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。