解析Adam在简单二次函数中的稳定性边界现象
Provable Edge-of-Stability for Adam on a One-Dimensional Quadratic
- 在单维二次函数上分析未修正Adam的动态行为
- 证明了Adam会趋向于恢复到特定稳定性阈值附近
- 揭示了该机制在某些条件下失效的情况
Adam的边缘稳定性（EoS）现象被广泛观测，但其背后的动态机制尚未完全明确。本文研究未修正的Adam在单维二次函数上的表现，该设定中恒定曲率使优化器自身的动力学与损失曲面变化分离。我们刻画了参数空间中的整体动态行为。在多数区域，证明了Adam表现出向其冻结稳定性阈值 $2(1+β_1)/[η(1-β_1)]$ 回归的倾向。同时识别出该边缘寻找机制失效的情形，包括严格亚临界周期轨道，以及特殊调参下持续超临界却收敛至最优解的轨迹。这些结果为无演化损失几何下的Adam EoS提供了具体的动力学解释，同时也暴露了其局限性。
原文摘要 · Abstract (English)
The edge-of-stability (EoS) phenomenon of Adam has been widely observed, while its underlying dynamical mechanism is not yet fully understood. We study uncorrected Adam on a one-dimensional quadratic, a clean setting where constant curvature isolates the optimizer-induced dynamics behind the EoS. We characterize the resulting dynamics across the parameter space. In broad regimes, we prove that Adam exhibits a restoring tendency toward its frozen stability threshold $2(1+β_1)/[η(1-β_1)]$. We also identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits and specially tuned trajectories that converge to the optimum while remaining uniformly supercritical. These results give a concrete dynamical explanation for Adam's EoS in a setting free of evolving loss geometry, while also exposing its limitations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。