用神经正切核分析cPIKANs的收敛性,揭示其优于PINNs的学习机制。
Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNs
- 通过神经正切核理论解析cPIKANs训练过程中的核结构演化。
- 发现cPIKANs在四类PDE上呈现可预测的谱特性与收敛行为。
- 首次系统揭示核行为与训练效率、域分解效果的关联,适合研究PDE求解器者参考。
物理信息柯尔莫戈罗夫-阿诺德网络(PIKANs)及其基于切比雪夫的变体(cPIKANs)近期成为求解偏微分方程(PDE)的有力工具。然而,其训练动态与收敛行为在理论上和数值上仍缺乏深入理解。本文利用神经正切核(NTK)理论,系统分析了标准cKANs在监督设置下的NTK,并扩展至物理信息场景。针对稳态亥姆霍兹方程、瞬态扩散与艾伦-卡恩方程、以及由欧拉-伯努利梁方程描述的受迫振动四类代表性PDE,我们研究了NTK矩阵的谱性质,包括特征值分布与谱偏差。同时考察了一阶、二阶及混合优化策略对NTK演化与学习动态的影响。结果表明,cPIKANs的NTK行为具有可解释性,其学习动态是传统物理信息神经网络(PINNs)无法捕捉的。谱趋势还揭示了域分解在特定条件下能提升训练效率,直接关联核行为与收敛速率。据我们所知,这是首个对cPIKANs进行系统的NTK研究,为理解并预测其实际性能提供了理论依据。
原文摘要 · Abstract (English)
Physics-informed Kolmogorov-Arnold Networks (PIKANs), and in particular their Chebyshev-based variants (cPIKANs), have recently emerged as promising models for solving partial differential equations (PDEs). However, their training dynamics and convergence behavior remain largely unexplored both theoretically and numerically. In this work, we aim to advance the theoretical understanding of cPIKANs by analyzing them using Neural Tangent Kernel (NTK) theory. Our objective is to discern the evolution of kernel structure throughout gradient-based training and its subsequent impact on learning efficiency. We first derive the NTK of standard cKANs in a supervised setting, and then extend the analysis to the physics-informed context. We analyze the spectral properties of NTK matrices, specifically their eigenvalue distributions and spectral bias, for four representative PDEs: the steady-state Helmholtz equation, transient diffusion and Allen-Cahn equations, and forced vibrations governed by the Euler-Bernoulli beam equation. We also conduct an investigation into the impact of various optimization strategies, e.g., first-order, second-order, and hybrid approaches, on the evolution of the NTK and the resulting learning dynamics. Results indicate a tractable behavior for NTK in the context of cPIKANs, which exposes learning dynamics that standard physics-informed neural networks (PINNs) cannot capture. Spectral trends also reveal when domain decomposition improves training, directly linking kernel behavior to convergence rates under different setups. To the best of our knowledge, this is the first systematic NTK study of cPIKANs, providing theoretical insight that clarifies and predicts their empirical performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。