用分组相对归一化降低零阶优化方差，提升大模型微调效率。
GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning
- 每样本仅一次扰动，通过分组相对归一化提升梯度方向数量
- 在不增加显存前提下，相比MeZO提升3.0%准确率，显存降低23%
- 适合资源受限场景下大模型微调，可替代现有零阶优化器
零阶（ZO）优化是微调大语言模型的低内存替代方案，但其部署受限于梯度估计的高方差。本文提出GRZO，一种分组相对零阶优化器，每个小批量样本仅采样一次伪独立扰动，并通过分组相对归一化聚合每个样本的损失，将有效梯度方向数从1提升至批次大小，且无需额外前向计算开销，同时保持推理级内存消耗。理论证明GRZO方向无偏，方差随批次大小缩小，非凸收敛界优于MeZO。在RoBERTa-large、Llama3-8B和OPT-13B多个任务上，GRZO使Llama3-8B平均准确率较MeZO提升+3.0%，峰值GPU显存降低23%；作为MeZO核心的即插即用替代品，显著提升稀疏、低秩及量化版本的零阶优化器性能，平均提升+6.0。
原文摘要 · Abstract (English)
Zeroth-order (ZO) optimization is a memory-efficient alternative to backpropagation for fine-tuning large language models, but its deployment is limited by the high variance of gradient estimation. We propose GRZO, a Group-Relative Zeroth-Order optimizer that draws one pseudo-independent perturbation per mini-batch example and aggregates the per-example losses through group-relative normalization, raising the effective gradient-direction count from one to the batch size at no additional forward cost while preserving inference-level memory. We prove that GRZO is directionally unbiased with variance shrinking proportionally to the batch size, yielding a tighter nonconvex convergence bound than MeZO. Across RoBERTa-large, Llama3-8B, and OPT-13B over multiple tasks, GRZO improves average accuracy on Llama3-8B by $+3.0$ over MeZO at $23\%$ lower peak GPU memory; as a drop-in replacement for the MeZO core, it lifts sparse, low-rank, and quantized ZO variants by $+6.0$ on average.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。