证明了凸函数下自适应梯度下降近线性收敛，方法更简洁。
A short proof of near-linear convergence of adaptive gradient descent under fourth-order growth and convexity
- 基于李雅普诺夫函数直接证明收敛性，避开复杂几何分析。
- 在唯一最小值点附近实现近线性收敛速度（接近1/√k）。
- 提出更自适应的变体算法，数值实验表现良好。
Davis、Drusvyatskiy 和 Jiang 证明了对于远离最小值处至少四次增长的光滑函数，带有自适应步长的梯度下降在局部以近乎线性速率收敛。该证明复杂，依赖于对所谓‘峡谷’流形上缓慢增长行为的监控。本文中，当目标函数为凸且具有唯一最小值时，我们提供了一种直接的李雅普诺夫型论证，避免了上述困难。作为副产品，我们得到了一个比原算法更具自适应性的改进版本，在数值实验中表现出色。
原文摘要 · Abstract (English)
Davis, Drusvyatskiy, and Jiang showed that gradient descent with an adaptive stepsize converges locally at a nearly-linear rate for smooth functions that grow at least quartically away from their minimizers. The argument is intricate, relying on monitoring the performance of the algorithm relative to a certain manifold of slow growth -- called the ravine. In this work, we provide a direct Lyapunov-based argument that bypasses these difficulties when the objective is in addition convex and a has a unique minimizer. As a byproduct of the argument, we obtain a more adaptive variant than the original algorithm with encouraging numerical performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。