提出新型确定性优化方法，实现加速收敛且理论可证。
Deterministic Adam-Inspired Methods with Accelerated Convergence Rate
- 解耦动量与自适应预条件，通过海森修正实现加速机制。
- 离散化后方法收敛率达 O(k⁻²)，优于传统Adam。
- 适合追求理论保证的优化算法研究者使用。
Adam 虽广泛使用，但其在确定性全批量设置下的收敛理论仍不完整，因动量与自适应预条件紧密耦合。针对光滑凸目标函数，本文通过变量与算子分裂分离动量变量，揭示了加速机制。随后结合海森驱动修正与基于梯度幅值的Adam式反馈，提出 Adam-HNAG（海森驱动的Nesterov加速梯度带Adam式自适应预条件）动态系统，其能量非负且指数衰减。该系统的离散化得到两种方法：Adam-HNAG 与同步变体 Adam-HNAG-s。在轨迹有界和一致性条件下，两者均满足离散李雅普诺夫收缩；若接受精确自适应步长，则目标值误差收敛率为 O(k⁻²)。数值实验验证了其行为特性。这些结果适用于所提方法，而非原始Adam递推。
原文摘要 · Abstract (English)
Adam is widely used, but its convergence theory remains incomplete even in the deterministic full-batch setting because momentum and adaptive preconditioning are tightly coupled. For smooth convex objectives, we split the momentum variable through variable-and-operator splitting, which reveals the acceleration mechanism. We then combine a Hessian-driven correction with Adam-style feedback based on the gradient magnitude. The resulting Adam-HNAG (Hessian-driven Nesterov accelerated gradient with Adam-style adaptive preconditioning) flow admits a nonnegative energy that decays exponentially. Its discretization yields two methods, Adam-HNAG and the synchronous variant Adam-HNAG-s. Under the stated trajectory-bound and consistency conditions, both methods satisfy a discrete Lyapunov contraction. If the exact adaptive steps are accepted, this contraction gives an $O(k^{-2})$ objective-value bound. Numerical experiments illustrate their behavior. These results apply to the proposed methods, not to the original Adam recursion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。