arXiv:2606.17013math.OCcs.LG2026-06
解析残差连接如何缓解深层网络梯度消失与爆炸问题
Exploding and vanishing gradients in deep neural networks: the effect of residual connections
- 基于乘性遍历理论分析梯度行为
- 残差连接可稳定梯度的李亚普诺夫谱
- 适合研究深度学习优化机制的学者
利用乘性遍历理论分析深度神经网络中梯度爆炸与消失现象。通过运用Furstenberg和Kifer关于李亚普诺夫指数的刻画，精确描述了李亚普诺夫谱的特性，并阐明了残差连接对此谱的影响。结果表明，残差结构能有效调控梯度传播的稳定性，为理解深层网络训练机制提供了理论依据。
原文摘要 · Abstract (English)
The well known phenomenon of exploding and vanishing gradients in deep neural networks is analyzed using multiplicative ergodic theory. The effect of adding a residual connection is explained in this context. Specifically, a characterization of Liapunov exponents due to Furstenberg and Kifer is exploited in order to make a precise statement about the Liapunov spectrum and the effect of residual connections on it.
深度学习梯度问题残差连接
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。