arXiv:2508.11522cs.LGhep-th2025-08被引 2

用费曼图简化神经网络有限宽度校正计算,提升训练动态预测精度。

Finite-Width Neural Tangent Kernels from Feynman Diagrams

  • 引入费曼图法计算神经网络有限宽度下的核函数修正项。
  • 在宽度大于20时,数值结果与实际网络采样统计一致。
  • 证明ReLU等不变尺度非线性函数无宽度修正,适用于深层网络分析。

神经切线核(NTK)是分析深度非线性神经网络的强大工具。在无限宽度极限下,大多数常见架构的NTK可被精确计算,实现对训练动态的完全解析控制。然而,在无限宽度下,训练中的关键特性如NTK演化或特征学习会消失。有限宽度效应可通过计算对无限宽度高斯统计的修正来纳入。本文提出用于计算NTK统计量有限宽度修正的费曼图方法,极大简化了代数运算,并实现了对前激活、NTK及特定高阶导数张量(dNTK和ddNTK)的逐层递推关系求解,从而在主导阶次上预测训练动态。通过将深层网络稳定性结果从前激活扩展至NTK,并证明对于如ReLU等尺度不变非线性函数,其NTK格拉姆矩阵对角线上无有限宽度修正,验证了框架可行性。我们数值实现了计算一阶修正所需的全部方程,结果显示当网络宽度n≳20时,结果与采样神经网络的统计行为高度吻合。

原文摘要 · Abstract (English)

Neural tangent kernels (NTKs) are a powerful tool for analyzing deep, non-linear neural networks. In the infinite-width limit, NTKs can easily be computed for most common architectures, yielding full analytic control over the training dynamics. However, at infinite width, important properties of training such as NTK evolution or feature learning are absent. Nevertheless, finite width effects can be included by computing corrections to the Gaussian statistics at infinite width. We introduce Feynman diagrams for computing finite-width corrections to NTK statistics. These dramatically simplify the necessary algebraic manipulations and enable the computation of layer-wise recursion relations for arbitrary statistics involving preactivations, NTKs and certain higher-derivative tensors (dNTK and ddNTK) required to predict the training dynamics at leading order. We demonstrate the feasibility of our framework by extending stability results for deep networks from preactivations to NTKs and proving the absence of finite-width corrections for scale-invariant nonlinearities such as ReLU on the diagonal of the Gram matrix of the NTK. We numerically implement the complete set of equations necessary to compute the first-order corrections for arbitrary inputs and demonstrate that the results follow the statistics of sampled neural networks for widths $n\gtrsim 20$.

神经网络核方法深度学习理论费曼图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。