通过稳定率调节，让梯度下降更偏向最优间隔解。
Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data
- 用动态稳定性控制梯度更新路径，实现对最大间隔解的隐式偏好。
- 理论证明迭代方向收敛到带屏障项的凸优化解，且有显式收敛速率。
- 适合研究优化器隐式偏差、高维分类问题的理论分析者阅读。
自适应梯度方法倾向于选择与标准梯度下降不同的最大间隔分离超平面，但固定正数值稳定性常数最终会再次改变更新几何结构。本文研究全批量线性分类在可分数据下的速率可控中间情形。对于无记忆的稳定性退火平滑符号下降（weighted exponential loss），我们证明归一化迭代序列收敛至一个凸Burg型屏障在间隔切片上的最小值点。证明将动力学精确重写为凹对偶目标上的熵镜面上升，通过KL递推控制对偶间隙，并得到显式的 $ S_t^{-1/2} $ 归一化迭代包络。静态屏障几何被完整刻画，包括KKT条件和两端极限。实验验证了对偶恒等式达到浮点误差级别，展示了预测路径与速率图，揭示累积时间中存在经验性的固定ε交叉缩放。进一步报告了逻辑尾部、固定ε交叉及自适应方法变体的鲁棒性与边界诊断，明确了所证平滑符号理论的适用范围。
原文摘要 · Abstract (English)
Adaptive gradient methods can favor max-margin separators that differ from gradient descent, yet a fixed positive numerical stability constant eventually changes the update geometry again. This paper studies the rate-controlled middle case for full-batch linear classification on separable data. For memoryless stability-annealed smoothed-sign descent with weighted exponential loss, we prove that the normalized iterates converge to the minimizer of a convex Burg-type barrier over a margin slice. The proof rewrites the dynamics exactly as entropic mirror ascent on a concave dual objective, controls the dual gap by a KL recursion, and yields an explicit S_t^{-1/2} normalized-iterate envelope. The static barrier geometry is fully characterized, including KKT conditions and both endpoint limits. Experiments validate the exact dual identities to floating-point error, illustrate the predicted path and rate diagram, and show an empirical fixed-epsilon crossover scaling in cumulative time. We further report robustness and boundary diagnostics for logistic tails, fixed-epsilon crossover, and adaptive-method variants, delineating the scope of the proved smoothed-sign theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。