通过自学习初始化解决物理神经网络收敛失败问题
LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks
- 设计门控分层优化框架，自动学习网络初始权重
- 在多个1D/2D/3D PDE场景中提升性能91.5%以上
- 适合需要稳定求解复杂偏微分方程的研究者
物理信息神经网络（PINNs）在偏微分方程（PDE）建模中影响广泛，但在复杂域或新域上常出现性能下降甚至收敛到平凡解。现有方法如超参数调优、课程学习或动态采样点重采样存在成本高、设计模糊或仍失效等问题。本文认为初始权重对训练崩溃有关键影响，但该方向研究不足。为此提出LIGO-PINN：通过门控分层优化实现网络权重的自学习初始化，以克服收敛失败。在1D、2D及挑战性2D流体动力学场景中评估表明，该方法优于现有先进方法，平均性能提升91.5%（覆盖六种基线），比最强基线提升81%。同时验证其可推广至3D非结构化域。进一步分析三类PDE的训练动态，揭示了传统PINNs失败机制与LIGO-PINN的优势来源。
原文摘要 · Abstract (English)
Physics-informed neural networks (PINNs) have had a broad research impact in modeling domains governed by partial differential equations (PDE). However, PINNs have been shown to perform poorly, sometimes even converging to trivial solutions, in challenging PDE domains, or when generalizing to unseen but related PDE domains. Previously proposed solutions detail hyperparameter tuning to reduce loss imbalance between data-driven and physics guided losses, curriculum learning based training strategies, or dynamic re-sampling of hard collocation points. These methods face certain pitfalls: hyperparameter tuning is expensive, designing a training curriculum is ambiguous in multi-parameter PDE settings, and dynamic resampling still fails in complex PDE settings. Complementary to this line of thinking, we believe the initial PINN network weights also play a crucial role in the emergence of catastrophic failures during training, yet the effect of PINN weight initialization has been surprisingly under-investigated. To this end, we propose a framework for Learned Initialization via Gated Layerwise Optimization (LIGO-PINN) to overcome PINN convergence failures. Through rigorous evaluation on 1D and 2D PDE domains, including a challenging 2D fluid dynamics setting, we demonstrate that our methodology outperforms state-of-the-art methods designed to alleviate PINN failures, achieving a 91.5% average performance improvement across six baselines and 81% over the strongest baseline. We also verify that LIGO-PINN generalizes to 3D unstructured domains. Finally, we analyze training dynamics across all three PDE domains to explain both LIGO-PINN's improvement and the convergence failure of traditional PINNs. Code: https://github.com/scailab/ligo-pinn Keywords: Machine Learning, Physics-Informed Neural Networks, Deep Learning, PDE Modeling
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。