提出预激活残差网络的集体核有效场理论，揭示其有限有效性窗口。
Collective Kernel EFT for Pre-activation ResNets
- 基于核仅闭包构建递归方程，推导出均值核、协方差和一阶修正项的连续深度微分系统。
- 均值核在所有深度保持准确，但协方差误差随深度累积至1阶量级。
- 发现核仅状态空间存在根本局限，需引入sigma核以改进精度，适合研究深层网络动力学者。
在有限宽度的深度神经网络中，经验核 $G$ 在层间随机演化。本文基于仅依赖 $G$ 的闭包层次，为预激活残差网络构建了集体核有效场理论（EFT），并诊断其有限的有效性窗口。利用残差增量的精确条件高斯性，我们推导出 $G$ 的精确随机递归关系。通过系统性地应用高斯近似，得到均值核 $K_0$、核协方差 $V_4$ 以及 $1/n$ 阶均值修正项 $K_{1, ext{EFT}}$ 的连续深度常微分方程组，其中 $K_{1, ext{EFT}}$ 图形上表现为一环自能修正。数值结果表明，$K_0$ 在所有深度均保持准确；但 $V_4$ 方程的残差随深度积累至 $O(1)$ 量级，主要源于 $G$-仅传输项的近似误差。此外，$K_{1, ext{EFT}}$ 失效，因源闭包在初始化阶段即存在系统性偏差。这些发现揭示了 $G$-仅状态空间约化的局限性，建议扩展状态空间以包含 sigma 核。
原文摘要 · Abstract (English)
In finite-width deep neural networks, the empirical kernel $G$ evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for pre-activation ResNets based on a $G$-only closure hierarchy and diagnose its finite validity window. Exploiting the exact conditional Gaussianity of residual increments, we derive an exact stochastic recursion for $G$. Applying Gaussian approximations systematically yields a continuous-depth ODE system for the mean kernel $K_0$, the kernel covariance $V_4$, and the $1/n$ mean correction $K_{1,\mathrm{EFT}}$, which emerges diagrammatically as a one-loop tadpole correction. Numerically, $K_0$ remains accurate at all depths. However, the $V_4$ equation residual accumulates to an $O(1)$ error at finite time, primarily driven by approximation errors in the $G$-only transport term. Furthermore, $K_{1,\mathrm{EFT}}$ fails due to the breakdown of the source closure, which exhibits a systematic mismatch even at initialization. These findings highlight the limitations of $G$-only state-space reduction and suggest extending the state space to incorporate the sigma-kernel.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。