arXiv:2412.05545cs.LGcs.DS2024-12被引 1

分析宽浅层神经算子梯度下降的收敛性,证明其可全局最优求解微分方程。

Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernel

  • 基于神经正切核框架,研究过参数化下浅层神经算子的训练过程。
  • 无论连续或离散时间,梯度下降均能线性收敛至全局最小值。
  • 为神经算子在科学计算中的可靠性提供理论支撑,适合相关研究者阅读。

神经算子旨在逼近函数空间之间的算子,在科学计算领域取得显著成果。相较于物理信息神经网络(PINNs)和深度里茨法(DRM)等深度学习求解器,神经算子可处理一类偏微分方程(PDEs)。尽管已有大量工作分析神经算子的逼近与泛化误差,但对其训练误差的分析仍不足。本文在神经正切核(NTK)框架下,对宽浅层神经算子及物理信息浅层神经算子的梯度下降进行收敛性分析。核心思想是:过参数化与随机初始化共同保证各权重向量在整个迭代过程中始终靠近初始值,从而实现梯度下降的线性收敛。本文证明,在过参数化设定下,无论连续时间或离散时间,梯度下降均可找到全局最小值。

原文摘要 · Abstract (English)

Neural operators are aiming at approximating operators mapping between Banach spaces of functions, achieving much success in the field of scientific computing. Compared to certain deep learning-based solvers, such as Physics-Informed Neural Networks (PINNs), Deep Ritz Method (DRM), neural operators can solve a class of Partial Differential Equations (PDEs). Although much work has been done to analyze the approximation and generalization error of neural operators, there is still a lack of analysis on their training error. In this work, we conduct the convergence analysis of gradient descent for the wide shallow neural operators and physics-informed shallow neural operators within the framework of Neural Tangent Kernel (NTK). The core idea lies on the fact that over-parameterization and random initialization together ensure that each weight vector remains near its initialization throughout all iterations, yielding the linear convergence of gradient descent. In this work, we demonstrate that under the setting of over-parametrization, gradient descent can find the global minimum regardless of whether it is in continuous time or discrete time.

神经算子收敛分析微分方程NTK

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。