用最优控制理论统一解释各种学习规则为何有效
Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules
- 把参数更新看作在损失曲面中导航的策略
- 梯度下降、动量等规则都是不同假设下的最优解
- 适合研究算法原理或设计自适应优化器的人
学习规则——即改进模型性能的参数更新方法——通常被假设而非推导。为什么某些规则比其他规则更有效？在什么条件下可认为某规则是最优的？本文提出一个理论框架，将学习规则视为在部分可观测的损失曲面中导航的策略，并将最优规则定义为相应最优控制问题的解。在不同假设下，一系列经典规则自然涌现：短期优化产生梯度下降，长期规划产生动量，考虑参数空间几何结构产生自然梯度，部分可控性导致非梯度规则，而Adam等自适应优化器则源于对损失曲面形状的在线贝叶斯推断。此外，持续学习中的权重重置可被理解为应对任务不确定性的最优响应。该框架通过单一目标统一多种现象，揭示了学习的计算结构，并为设计自适应算法提供原则性基础。
原文摘要 · Abstract (English)
Learning rules -- prescriptions for updating model parameters to improve performance -- are typically assumed rather than derived. Why do some learning rules work better than others, and under what assumptions can a given rule be considered optimal? We propose a theoretical framework that casts learning rules as policies for navigating (partially observable) loss landscapes, and identifies optimal rules as solutions to an associated optimal control problem. A range of well-known rules emerge naturally within this framework under different assumptions: gradient descent from short-horizon optimization, momentum from longer-horizon planning, natural gradients from accounting for parameter space geometry, non-gradient rules from partial controllability, and adaptive optimizers like Adam from online Bayesian inference of loss landscape shape. We further show that continual learning strategies like weight resetting can be understood as optimal responses to task uncertainty. By unifying these phenomena under a single objective, our framework clarifies the computational structure of learning and offers a principled foundation for designing adaptive algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。