改进权重平均方法，让语言模型微调更稳定高效
EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes
- 提出修正偏差的指数移动平均（BEMA），消除传统EMA的滞后问题
- 在多个语言模型基准上，收敛速度和最终性能均优于EMA和普通训练
- 理论可证明加速效果，适合追求稳定高效的微调研究者
语言模型微调中因小批量导致的随机性会引发生成质量波动，破坏训练稳定性。常用方法是使用指数移动平均（EMA）平滑权重，但会引入旧迭代值的偏差，造成优化滞后。本文提出偏置修正的指数移动平均（BEMA），在保留方差抑制优势的同时消除偏差。基于一个简单理论模型，我们证明了BEMA在理论上优于标准EMA和普通训练。在多种标准语言模型基准上的大量实验表明，BEMA显著提升了收敛速度与最终性能，是一种兼具理论依据与实用价值的微调优化方案。
原文摘要 · Abstract (English)
Stochasticity in language model fine-tuning, often caused by the small batch sizes typically used in this regime, can destabilize training by introducing large oscillations in generation quality. A popular approach to mitigating this instability is to take an Exponential moving average (EMA) of weights throughout training. While EMA reduces stochasticity, thereby smoothing training, the introduction of bias from old iterates often creates a lag in optimization relative to vanilla training. In this work, we propose the Bias-Corrected Exponential Moving Average (BEMA), a simple and practical augmentation of EMA that retains variance-reduction benefits while eliminating bias. BEMA is motivated by a simple theoretical model wherein we demonstrate provable acceleration of BEMA over both a standard EMA and vanilla training. Through an extensive suite of experiments on Language Models, we show that BEMA leads to significantly improved convergence rates and final performance over both EMA and vanilla training in a variety of standard LM benchmarks, making BEMA a practical and theoretically motivated intervention for more stable and efficient fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。