为优化器设计提供物理机制解释,提出可自适应记忆长度的新算法。
A Physical Response-and-Memory Model for Muon Optimization

- 将权重更新视为有记忆的响应介质,用物理模型解释梯度方向选择原理。
- 提出双时间尺度记忆机制的Bi-Maxwell优化器,显著减少达到目标损失的训练步数。
- 适用于大模型训练场景,对理解优化器设计本质具有理论价值。
训练大型语言模型成本高昂,相同算力下最终能到达的损失值取决于每一步梯度如何转化为权重更新——这一转换规则即为优化器。从SGD和AdamW到最近的Muon,有效更新规则大多依赖工程直觉,并通过基准测试选择。Muon在更新前对动量矩阵进行半正交化,持续刷新公开训练基准记录;然而为何半正交化方向有效,以及动量应平均多长历史,目前仍主要依赖经验解答。本文将训练过程中的权重矩阵视为具有记忆的响应介质,构建其物理模型:半正交化方向是在输出端安全预算下的最大耗散响应,解释了其有效性;动量是介质内部累积的应力;应力松弛时间决定了平均时长,真实介质存在多个松弛时间尺度,最简形式为一个快一个慢。基于此,提出Bi-Maxwell优化器。该框架还得出可验证推论:训练早期梯度方向变化快,后期变慢,因此最优记忆长度应随训练阶段增长;通过对8条独立训练轨迹的只读探测代理测量,结果与此一致。仅将记忆核从单时间尺度改为双时间尺度,即可在公开的大语言模型优化器基准上显著减少达到目标损失的步数。
原文摘要 · Abstract (English)
Training large language models is costly. How low a loss the same compute can ultimately reach depends on how each step's gradient is converted into a weight update; the rule that performs this conversion is the optimizer. From SGD and AdamW to the recent Muon, effective update rules have mostly been shaped by engineering intuition and then selected on benchmarks. Muon semi-orthogonalizes the momentum matrix before applying the update and has kept breaking records on public training benchmarks; yet why the semi-orthogonalized direction works, and over how long a history the momentum should average, are two questions at present answered mainly by experience. Here we treat the weight matrix during training as a responsive medium with memory and build a physical model for it, in which both questions find answers: the semi-orthogonalized direction is the maximally dissipative response under an output-side safety budget, which explains why it works; momentum is the internal stress accumulated by the medium; how long it should average is set by the relaxation of this stress, and a real medium relaxes on more than one timescale, the simplest form being one fast and one slow. On this basis we propose the Bi-Maxwell optimizer. The framework further yields a testable consequence: gradient directions change fast early in training and more slowly later, so the optimal memory length should grow with training stage; step-by-step measurements of a proxy for it by a read-only probe across 8 independent training trajectories are consistent with this consequence. Replacing the memory kernel alone, from a single timescale to two, brings training to the target loss in noticeably fewer steps on a public large-language-model optimizer benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。