证明了宽两层物理神经网络用随机梯度下降可线性收敛
Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation
- 采用广义激活函数的宽两层物理神经网络
- 在高概率意义下实现线性收敛，解决泊松方程
- 揭示了随机优化中矩阵正定性的关键作用
物理信息神经网络（PINNs）是求解偏微分方程的热门方法。实际中常使用随机梯度下降类算法训练网络，因此其收敛性至关重要。本文建立了在高概率意义下，对一类通用激活函数的过参数化两层PINNs训练时，随机梯度下降/流的线性收敛性，用于求解泊松方程这一典型的二阶椭圆问题。该结果扩展了先前仅分析梯度下降的工作。分析难点在于处理随机优化引入的动态随机性，核心在于保证训练过程中合适格拉姆矩阵的正定性。研究揭示了优化过程的动力学机制，并为随机算法训练的物理信息神经网络提供了理论保障。
原文摘要 · Abstract (English)
Physics informed neural networks (PINNs) represent a very popular class of neural solvers for partial differential equations. In practice, one often employs stochastic gradient descent type algorithms to train the neural network. Therefore, the convergence guarantee of stochastic gradient descent is of fundamental importance. In this work, we establish the linear convergence of stochastic gradient descent / flow in training over-parameterized two layer PINNs with a general class of activation functions for solving one model second-order elliptic problem, i.e., the Poisson equation, in the sense of high probability. These results extend the existing result [20] in which gradient descent was analyzed. The challenge of the analysis lies in handling the dynamic randomness introduced by stochastic optimization methods. The key of the analysis lies in ensuring the positive definiteness of suitable Gram matrices during the training. The analysis sheds insight into the dynamics of the optimization process, and provides guarantees on physics informed neural networks trained by stochastic algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。