通过投影梯度对齐扰动，加速大模型零阶微调。
Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations
- 在低维梯度空间中对齐扰动方向，降低方差。
- 分类任务最高提升6%准确率，生成任务提升12%。
- 训练迭代减少81%，显存消耗降低70%，适合资源受限场景。
使用零阶（ZO）优化进行大语言模型（LLMs）微调因其较低的内存需求而成为传统基于梯度方法的有前景替代方案。然而，现有ZO方法存在梯度估计方差高、收敛慢、大规模模型性能不佳的问题。本文提出P-GAP，一种基于投影梯度对齐扰动的快速零阶微调方法。首先估计低维梯度空间，并在该空间内对齐扰动方向，从而减少扰动参数数量并降低方差，实现更快收敛。实验表明，P-GAP在多个大模型上持续优于基线方法，在分类任务中准确率最高提升6%，生成任务提升12%，训练迭代次数减少约81%，GPU耗时降低70%。结果证明，P-GAP实现了快速、可扩展且资源高效的零阶大模型微调。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) using zeroth-order (ZO) optimization has emerged as a promising alternative to traditional gradient-based methods due to its reduced memory footprint requirement. However, existing ZO methods suffer from high variance in gradient estimation, leading to slow convergence and suboptimal performance on large-scale models. In this work, we propose P-GAP, a fast LLM fine-tuning approach through zeroth-order optimization with Projected Gradient-Aligned Perturbations. Specifically, we first estimate a low-dimensional gradient space and then align perturbations in projected gradients' direction within the space. This approach enables reduced the number of perturbed parameters and decreased variance, therefore accelerated convergence for LLM fine-tuning. Experiments on LLMs show that P-GAP consistently surpasses the baselines, achieving up to 6% increase in accuracy on classification tasks and up to 12% higher accuracy on generation tasks, with up to about 81% less training iterations and 70% less GPU hours. These results demonstrate that P-GAP enables fast, scalable, and resource-efficient ZO LLM fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。