提出新微分方程模型，更准确描述大步长梯度下降行为
Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability
- 将梯度下降看作一根'杆'的连续运动，构建物理可解释的微分方程
- 在简单模型和神经网络上均逼近效果优于或等于现有方法
- 计算高效且能解释模型自稳定现象，适合研究训练动态的学者
理解非凸景观下基于梯度的训练机制仍具挑战。Cohen 等人（2021）提出的稳定性边缘现象表明，大步长梯度下降（GD）常偏离梯度流。近期提出的中央流（Central Flow）在多种架构上提供了对 GD 动态的精确常微分方程（ODE）近似。本文提出一种替代性 ODE 模型——杆流（Rod Flow），具有以下优势：（1）基于将 GD 迭代视为一维扩展物体（“杆”）的物理图像，推导过程具有理论根基；（2）在简单玩具模型中更准确捕捉 GD 动态，且在代表性神经网络架构上的精度与中央流相当；（3）为显式形式，计算成本低。理论上，我们证明杆流能正确预测临界尖锐度阈值，并解释四次势能中的自稳定现象。通过一系列数值实验验证了理论结论。
原文摘要 · Abstract (English)
How can we understand gradient-based training over non-convex landscapes? The edge of stability phenomenon, introduced in Cohen et al. (2021), indicates that the answer is not so simple: namely, gradient descent (GD) with large step sizes often diverges away from the gradient flow. In this regime, the "Central Flow", recently proposed in Cohen et al. (2025), provides an accurate ODE approximation to the GD dynamics over many architectures. In this work, we propose Rod Flow, an alternative ODE approximation, which carries the following advantages: (1) it rests on a principled derivation stemming from a physical picture of GD iterates as an extended one-dimensional object -- a "rod"; (2) it better captures GD dynamics for simple toy examples and matches the accuracy of Central Flow for representative neural network architectures, and (3) is explicit and cheap to compute. Theoretically, we prove that Rod Flow correctly predicts the critical sharpness threshold and explains self-stabilization in quartic potentials. We validate our theory with a range of numerical experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。