用连续动力学建模训练过程，预测权重变化更准更快
Gradient Flow Matching for Learning Update Dynamics in Neural Network Training
- 将训练看作可学习的动态系统，用向量场捕捉优化器行为
- 从部分训练数据预测最终权重，准确率媲美Transformer
- 适用于多种网络结构，适合研究优化机制与加速收敛
深度神经网络训练因基于梯度的迭代优化而计算成本高。我们提出梯度流匹配（GFM），一种连续时间建模框架，将神经网络训练视为由学习到的、针对优化器的向量场驱动的动力系统。通过条件流匹配，GFM捕获SGD、Adam、RMSprop等优化器的底层更新规则，实现权重轨迹向收敛点的平滑外推。与黑箱序列模型不同，GFM在学习目标中融入了基于梯度更新的结构知识，从而能从部分训练序列准确预测最终权重。实验证明，GFM的预测精度与基于Transformer的模型相当，显著优于LSTM等经典基线。此外，GFM在不同神经网络架构和初始化下具有泛化能力，提供了一个统一框架，用于研究优化动力学并加速收敛预测。
原文摘要 · Abstract (English)
Training deep neural networks remains computationally intensive due to the itera2 tive nature of gradient-based optimization. We propose Gradient Flow Matching (GFM), a continuous-time modeling framework that treats neural network training as a dynamical system governed by learned optimizer-aware vector fields. By leveraging conditional flow matching, GFM captures the underlying update rules of optimizers such as SGD, Adam, and RMSprop, enabling smooth extrapolation of weight trajectories toward convergence. Unlike black-box sequence models, GFM incorporates structural knowledge of gradient-based updates into the learning objective, facilitating accurate forecasting of final weights from partial training sequences. Empirically, GFM achieves forecasting accuracy that is competitive with Transformer-based models and significantly outperforms LSTM and other classical baselines. Furthermore, GFM generalizes across neural architectures and initializations, providing a unified framework for studying optimization dynamics and accelerating convergence prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。