针对大模型训练中的噪声问题，提出高效分布式优化框架TailOPT
Efficient Distributed Optimization under Heavy-Tailed Noise
- 通过自适应优化与坐标剪裁双重机制应对重尾噪声
- 在梯度方差无界条件下仍保证收敛性，支持本地更新
- 轻量级实现Bi²Clip，无需传递额外统计量，适合大规模训练
分布式优化已成为现代机器学习中因模型和数据规模扩大而默认的训练范式。为降低通信开销，常在全局聚合前进行本地更新，形成内层与外层嵌套优化。然而，注意力模型中普遍存在的重尾随机梯度噪声仍是主要挑战，严重影响训练效果。本文提出TailOPT框架，通过自适应优化或剪裁技术有效应对重尾噪声。我们建立了在梯度方差可能无界的重尾噪声下，带有本地更新的TailOPT框架的收敛性理论。其中，我们重点介绍一种内存与通信高效的实例化方法——Bi²Clip，该方法在内层与外层优化器上均进行坐标级剪裁，实现类似Adam的自适应性能，但无需维护或传输额外梯度统计信息。实验表明，包括Bi²Clip在内的TailOPT在多个语言任务和模型上表现优于现有最先进方法。
原文摘要 · Abstract (English)
Distributed optimization has become the default training paradigm in modern machine learning due to the growing scale of models and datasets. To mitigate communication overhead, local updates are often applied before global aggregation, resulting in a nested optimization approach with inner and outer steps. However, heavy-tailed stochastic gradient noise remains a significant challenge, particularly in attention-based models, hindering effective training. In this work, we propose TailOPT, an efficient framework designed to address heavy-tailed noise by leveraging adaptive optimization or clipping techniques. We establish convergence guarantees for the TailOPT framework under heavy-tailed noise with potentially unbounded gradient variance and local updates. Among its variants, we highlight a memory and communication efficient instantiation which we call $Bi^2Clip$, which performs coordinate-wise clipping at both the inner and outer optimizers, achieving adaptive-like performance (e.g., Adam) without the cost of maintaining or transmitting additional gradient statistics. Empirically, TailOPT, including $Bi^2Clip$, demonstrates superior performance on several language tasks and models, outperforming state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。