揭示平均Adam优化器的收敛速度与分布规律
Central limit theorem for the averaged Adam optimizer
- 从动力学角度分析平均Adam收敛至吸引子的机制
- 证明收敛速度为n⁻¹/²，与经典随机逼近一致
- 给出收敛分布的协方差表达式，适用于理论研究者
本文分析了平均Adam优化器收敛至Adam向量场吸引子的过程，并给出了中心极限定理。该定理精确刻画了算法的收敛速度，其阶数为n⁻¹/²（n为迭代步数），与经典随机逼近算法一致。中心极限定理中的协方差由吸引子状态下的Adam算法特性决定。
原文摘要 · Abstract (English)
In this article, we analyse convergence of the averaged Adam optimizer to an attracting zero of the Adam vector field. We provide a central limit theorem that, in particular, quantifies exactly the speed of convergence. The order of convergence is $n^{-1/2}$ in the number of steps of the algorithm which coincides with the order observed for classical stochastic approximation algorithms. The covariance in the central limit theorem is given in terms of properties of the Adam algorithm in the state of the attractor.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。