提出DNTK理论，证明PINN在无限宽时的核正定性。
The Differential Neural Tangent Kernel and Its Positivity
- 构建DNTK框架，通过微分算子扩展NTK分析PINNs
- 证明任意线性微分算子下，浅层与深层网络的DNTK正定
- 适用于RePU及光滑非多项式激活函数，为训练分析奠基
神经正切核（NTK）是分析过参数化神经网络训练动态的强大工具。近期，该理论被拓展至物理信息神经网络（PINNs）以求解线性偏微分方程，此类网络是主流的神经微分方程求解器。在此分析中，相关NTK的正定性起着关键作用。然而，由于存在多个微分算子，建立PINNs的NTK正定性极具挑战。本文提出一种新理论框架——微分神经正切核（DNTK），通过NTK视角分析PINNs，证明了在无限宽度下，对于包括RePU和光滑非多项式激活函数在内的广泛激活函数，以及所有线性微分算子，浅层与深层神经网络的DNTK均保持正定。这些理论结果为梯度类算法训练PINNs的分析奠定了基础。
原文摘要 · Abstract (English)
The Neural Tangent Kernel (NTK) is one powerful tool for analyzing the training dynamics of neural networks in the over-parameterized regime. Recently, the theoretical framework has been extended to physics-informed neural networks (PINNs) for solving linear PDEs, one highly popular class of neural PDE solvers. In the analysis, the positivity of the associated NTK plays a fundamental role. However, establishing the positivity of the NTK for PINNs is highly challenging, due to the presence of multiple differential operators. In this work, we propose a new theoretical framework, called Differential Neural Tangent Kernel (DNTK), for analyzing PINNs through the lens of the NTK, and establish the positivity of the infinite width DNTK for both shallow and deep neural networks for a wide class of activation functions, including RePU and smooth but non-polynomial activations, for all linear differential operators. These theoretical results lay the foundation for the analysis of gradient type algorithms for training PINNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。