用规范场论视角解析神经网络层间信息传播机制
Deep neural networks as lattice gauge theories
- 将神经网络每层视为晶格点，权重矩阵对应规范场，构建$(0+1)$维场论模型
- 通过微扰展开计算层间传播，发现$O(1/N)$阶的无限圈图贡献统计涨落
- 首次建立神经元散射振幅框架，可用于分析深层网络高阶相关性
我们改进了原有的神经网络/量子场论对偶关系，引入网络各层的置换对称性，构建出一个$(0+1)$维晶格规范场论：每层$N$个神经元构成$N$分量晶格点，权重矩阵则作为连接线上的规范场。在此框架下，我们计算了树级神经元-神经元传播子，描述网络中层方差的演化，并发展出费曼图方法以在$1/N$微扰展开中计算相互作用。特别地，我们得到了$O(1)$阶修正的递归表达式，表征了网络集合中的统计涨落，包含来自前层的无限多圈图介导的相互作用。此外，我们初步分析了按$1/N$阶展开的神经元散射振幅，为研究深层网络的高阶相关性和信息传播提供了场论框架。最后，我们讨论了神经网络与量子场论交叉领域的若干有趣未来方向。
原文摘要 · Abstract (English)
We modify the NN/QFT duality [1] to incorporate the layerwise permutation symmetry of the network, resulting in a $(0\!+\!1)$-dimensional lattice gauge theory, in which each layer of $N$ neurons acts as an $N$-component lattice site, and the weight matrices play the role of gauge fields living on the links. In this framework, we compute the tree-level neuron-neuron propagator which describes the evolution of layer variance in the network, and develop the Feynman diagram machinery to compute interactions in the perturbative expansion in $1/N$. In particular, we obtain a recursive expression for all corrections to the exact propagator at $O(1)$, representing statistical fluctuations in the ensemble of networks, including infinitely-many loop diagrams mediating the interactions from previous layers. We also present a preliminary analysis of neuron scattering amplitudes that contribute order-by-order in $1/N$, which provides a field-theoretic framework for studying higher-point correlations, and by extension information propagation, in deep networks. We remark on some interesting directions for future work at the intersection of neural networks and quantum field theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。