针对大模型训练中梯度爆炸问题，提出分组自适应裁剪方法，提升训练稳定性与效果。
AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
- 按功能模块分组参数，用指数移动平均动态调整裁剪阈值。
- 在GSM8K上使Mistral-7B准确率达72.93%，优于LoRA的69.5%。
- 轻量设计可无缝接入现有训练流程，适合高效微调场景。
为稳定大语言模型（LLMs）训练，梯度裁剪是缓解梯度爆炸的常用方法。然而，传统全局范数裁剪错误假设不同功能模块间梯度同质，导致波动参数拖累稳定参数的过度缩放，产生不利的“溢出效应”。为此，本文提出自适应分组梯度裁剪（AGGC），根据功能类型将参数分组，并利用指数移动平均（EMA）基于历史行为调节每组。具体而言，构建自适应区间以同时抑制梯度爆炸与消失，并采用时间依赖调度机制平衡探索与收敛。在LLaMA 2-7B、Mistral-7B和Gemma-7B上实验表明，AGGC持续优于LoRA，且常胜过全量微调。在GSM8K基准上，使用AGGC微调的Mistral-7B准确率达72.93%，高于LoRA的69.5%。AGGC还能有效稳定带可验证奖励的强化学习（RLVR），提升Qwen 2.5与Llama 3.2的逻辑推理能力。实验表明，AGGC通过模块化自适应裁剪策略，有效克服传统方法对梯度异质性的局限。因其轻量设计，可无负担集成至现有后训练流程。
原文摘要 · Abstract (English)
To stabilize the training of Large Language Models (LLMs), gradient clipping is a nearly ubiquitous heuristic used to alleviate exploding gradients. However, traditional global norm clipping erroneously presupposes gradient homogeneity across different functional modules, leading to an adverse "spill-over" effect where volatile parameters force unnecessary scaling on stable ones. To overcome this, we propose Adaptive Group-wise Gradient Clipping (AGGC). AGGC partitions parameters into groups based on functional types and regulates each according to its historical behavior using an Exponential Moving Average (EMA). Specifically, it constructs an adaptive interval to simultaneously mitigate gradient explosion and vanishing, while employing a time-dependent scheduling mechanism to balance exploration and convergence. Experiments on LLaMA 2-7B, Mistral-7B, and Gemma-7B models show that AGGC consistently outperforms LoRA and frequently surpasses Full Fine-Tuning. On the GSM8K benchmark, Mistral-7B fine-tuned with AGGC achieves an accuracy of 72.93%, exceeding LoRA's 69.5%. AGGC also effectively stabilizes Reinforcement Learning with Verifiable Rewards (RLVR), enhancing the logic deduction of Qwen 2.5 and Llama 3.2 models. Experimental results demonstrate that AGGC effectively addresses the limitations of traditional gradient clipping methods, particularly in overcoming gradient heterogeneity, by utilizing a modular, adaptive clipping strategy to stabilize the training process. Due to its lightweight design, AGGC can be seamlessly integrated into existing post-training pipelines with negligible overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。