提出新方法缓解物理神经网络训练中的梯度冲突问题。
Mitigating Gradient Pathology in PINNs through Aligned Constraint
- 将零阶项重构成对齐约束，减少梯度冲突。
- 引入延迟因子避开高曲率区域，提升优化效率。
- 适用于复杂几何的PDE求解，尤其适合高难度问题。
物理信息神经网络（PINNs）在求解偏微分方程（PDEs）方面具有强大能力，但其训练常因梯度病理而陷入停滞。来自PDE残差与边界约束的梯度相互抵消，导致模型困于局部极小值。现有方法如自适应加权或硬约束，或无法根本解决病态问题，或仅限于简单几何。本文从损失景观与优化动态角度系统分析梯度病理成因，提出约束对齐与流形提升（CAML）方法。通过将所有零阶项重构为对齐约束，有效缓解梯度冲突；同时引入延迟因子，帮助优化器跳过高曲率区域。实验表明，CAML显著提升了复杂PINN问题中的数值稳定性和求解效率。代码已开源：https://github.com/YichenLuo-0/CAML。
原文摘要 · Abstract (English)
While Physics-Informed Neural Networks (PINNs) are powerful for solving Partial Differential Equations (PDEs), their training is often paralyzed by gradient pathology. The gradients from the PDE residuals and boundary constraints oppose each other, trapping the model in local minima. Current solutions, such as adaptive weighting or hard constraints, either fail to fundamentally resolve this ill-conditioning or are limited to simple geometries. In this study, we systematically analyze the possible causes of this gradient pathology from the perspectives of loss landscapes and optimization dynamics. Based on the obtained conclusion, we propose Constraint-Aligned loss with Manifold Lifting (CAML). By reformulating all zeroth-order terms into aligned constraints, our method effectively mitigates gradient conflicts. In addition, we introduce a delay factor to help the optimizer skip the high-curvature area. Experiments demonstrate that our CAML significantly enhances numerical stability and efficiency in highly complex PINN problems. Our code is open-sourced on https://github.com/YichenLuo-0/CAML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。