用物理模型设计自适应动量，训练更快且不需调参。
Critical Damping as a Momentum Schedule: Multi-Seed Validation, a Hybrid Recipe, and an Exhaustive Negative Result on Surgical Layer Selection
- 基于阻尼振荡器理论推导出无需调参的动量调度策略。
- 在5个随机种子上比固定动量快2.34倍，最终准确率低0.46个百分点。
- 混合方案可消除准确率损失，适合追求训练速度的场景。
基于带动量的SGD的阻尼谐振子模型，临界阻尼条件导出无须调参的动量调度：mu(t) = 1 - 2*sqrt(alpha(t))。在ResNet-18/CIFAR-10（200轮余弦学习率）上，该方法在5个随机种子中均以2.34倍速度达到90%测试准确率（范围1.71–2.86倍，单侧配对t检验p=4e-4），但最终准确率下降0.46个百分点（5/5种子，p=0.009）。短周期对照实验排除了训练长度偏差的影响：压缩基线要么牺牲0.5–0.9个百分点准确率，要么在相同准确率下仍更慢。采用混合策略——前段使用临界阻尼动量至90%准确率，后段切换为常数mu=0.9——可消除准确率损失，最终准确率达95.45±0.05%，且到达90%的速度提升2.4倍（n=5）。该加速效果在其他架构（如无跳跃连接的VGG-16）上也成立（1.72倍，n=3）；在CIFAR-100上早期优势持续（2–4倍至中段阈值），但准确率代价增至-1.7个百分点，使等准确率下的加速降至1.14倍。此外，本文报告了一项彻底的负面结果：对误分类图像进行梯度归因，原声称可指导重训层选择，但在全部35种三组层组合中，所选组合排名仅第11位（精确p=0.31），表现不优于随机。幸存的是弱但真实的信号：包含梯度归因最高分层的组合平均误差减少6.2，而梯度范数最低的层始终导致最严重损害（最多-20个错误）。梯度归因是避害信号，而非修复目标。本文公开完整35组合结果作为层选择研究的基线。
原文摘要 · Abstract (English)
The critical damping condition of the damped harmonic oscillator model of SGD with momentum (Qian, 1999) yields a momentum schedule with no tuned hyperparameters: mu(t) = 1 - 2*sqrt(alpha(t)). Across five seeds on ResNet-18/CIFAR-10 (200-epoch cosine schedule) it reaches 90% test accuracy 2.34x faster than constant mu=0.9 (range 1.71-2.86x, 5/5 seeds, one-sided paired t-test p=4e-4), at the cost of a real final-accuracy deficit of 0.46 pp (5/5 seeds, p=0.009). A short-schedule control rules out a schedule-length artifact: compressed baselines either pay 0.5-0.9 pp of accuracy or stay slower to 90% at equal accuracy. A hybrid recipe -- critical-damping momentum until 90%, then constant mu=0.9 -- removes the deficit and keeps the speedup: 95.45 +/- 0.05% final accuracy at 2.4x faster progress to 90% (n=5). The speedup generalizes across architectures (VGG-16 without skip connections: 1.72x, n=3); on CIFAR-100 early gains persist (2-4x to mid-training thresholds) but the accuracy cost grows (-1.7 pp), compressing the accuracy-matched gain to 1.14x. We also report an exhaustive negative result on surgical layer selection. Version 2 of this paper claimed that gradient attribution on misclassified images selects which layers to retrain; running the identical correction protocol on all 35 combinations of 3-of-7 layer groups ranks the selected triple 11th of 35 (exact p=0.31) -- no better than random. What survives is weaker but real: combinations containing the top-ranked layer outperform the rest (+6.2 vs -1.4 mean net error reduction), and the bottom of the gradient-norm ranking reliably predicts the most harmful interventions (down to -20 net errors). Gradient attribution on errors is a harm-avoidance signal, not a selector of repair targets. We release the full 35-combination landscape as a baseline for layer-selection claims.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。