解释了学习率降温阶段为何有时有效、有时无效。
Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps
- 通过梯度噪声结构与优化器是否归一化共同判断降温是否有益。
- 在乘性噪声下,SGD自动衰减,降温无用;归一化方法有噪声底限,降温可提升性能。
- 适用于理解大模型训练中学习率调度的机制,适合优化算法研究者。
大规模模型预训练中,学习率热身-稳定-衰减(WSD)调度的降温阶段在某些情况下降低最终损失,而在其他情况下则无效。本文给出了一个可证明的解释:这取决于两个因素的共同作用——梯度噪声的结构和优化器是否对更新进行归一化。在强凸目标函数与乘性(与梯度成比例)噪声条件下,随机梯度下降(SGD)以恒定学习率几何收缩,因此降温无改进空间。而在此相同设定下,基于符号和归一化的优化方法(自适应优化器的标准代理),其收敛到噪声底限量级为 $η^2$,仅当学习率趋于零时才能逼近最小值;任何加性噪声均会重新引入各方法的底限。核心机制在于:SGD步长随梯度缩小,自我衰减;归一化步长保持单位尺度,无法衰减。本文精确求解了二次目标下的 signSGD 平稳律,给出底限常数的闭式表达,证明了在 $(L_0,L_1)$-光滑条件下的局部解耦性,通过尺度不变性将底限扩展至 d>1 的归一化 SGD,且验证了动量与重尾噪声下的鲁棒性。模拟结果完全符合预测,并在真实分类任务上利用直接测量的梯度噪声验证了噪声区间诊断。该机制解释了降温是否有效;实际大规模训练中使用的内部降温比例,已超出平稳景观与噪声几何的范围。
原文摘要 · Abstract (English)
The cooldown phase of a warmup-stable-decay (WSD) learning-rate schedule, now a default in large-model pretraining, lowers the final training loss in some settings and does nothing in others. We give a provable account of which case obtains, and it turns on two properties together: the structure of the gradient noise and whether the optimizer normalizes its update. On a strongly convex objective with multiplicative (gradient-proportional) noise, stochastic gradient descent contracts geometrically at a constant learning rate, so cooldown has nothing to improve. Under the same objective and noise, sign-based and normalized methods, the standard surrogates for adaptive optimizers, settle on a noise floor of order $η^2$ and reach the minimizer only as the learning rate is driven to zero; any additive noise then reinstates a floor for every method. The mechanism is elementary: an SGD step shrinks in proportion to the gradient and so anneals itself, whereas a normalized step keeps unit scale and cannot. We solve the signSGD stationary law on the quadratic exactly and obtain the floor constant in closed form, prove a local form of the dissociation under $(L_0,L_1)$-smoothness, extend the floor to normalized SGD in dimension d>1 by a scale-invariance argument, and establish robustness to momentum and heavy-tailed noise. Simulation confirms every prediction, and we demonstrate the resulting noise-regime diagnostic on a real classification task with directly measured gradient noise. The mechanism explains whether cooldown helps; the interior cooldown fraction used at scale lies outside stationary landscape-and-noise geometry.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。