通过随机理论揭示深度网络中涌现的级联行为规律
Tuning Universality in Deep Neural Networks
- 引入中心极限定理级波动，建立深度信息传播的随机理论
- 四组有效耦合参数决定级联行为，可调控至两种不同普适类
- 激活函数设计能控制随机深度网络的集体动力学特性
深度神经网络（DNN）表现出类似裂纹的级联活动，其机制尚不明确。本文通过引入中心极限定理级别的波动，推导出深度信息传播（DIP）的随机理论。四个有效耦合参数（r, h, D₁, D₂）刻画系统动态，得到静态指数的朗道描述和活动级联的定向渗流（DP）结构。调节这些耦合参数可使级联行为在对数势阱中的布朗运动与吸收自由布朗运动之间切换，分别对应不同的普适类。数值模拟验证了理论，并表明激活函数设计能调控随机深度神经网络中的集体动力学。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) exhibit crackling-like avalanches whose origin lacks a mechanistic explanation. Here, I derive a stochastic theory of deep information propagation (DIP) by incorporating Central Limit Theorem (CLT)-level fluctuations. Four effective couplings $(r, h, D_1, D_2)$ characterize the dynamics, yielding a Landau description of the static exponents and a Directed Percolation (DP) structure of activity cascades. Tuning the couplings selects between avalanche dynamics generated by a Brownian Motion (BM) in a logarithmic trap and an absorbed free BM, each corresponding to a distinct universality classes. Numerical simulations confirm the theory and demonstrate that activation function design controls the collective dynamics in random DNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。