提出无需反向传播的低精度微调方法，提升大模型在4/8位下的性能
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
- 基于零阶优化，仅用前向传播实现低精度微调
- 在INT4/INT8下表现优于传统一阶方法，GLUE任务性能相当
- 适用于资源受限场景，尤其适合部署时的高效微调
大型语言模型常通过量化至低精度（如4或8位）以降低推理时的内存占用和延迟。然而量化会损害模型性能，需进行微调。传统微调方法如随机梯度下降和Adam依赖反向传播，在低精度下易出错。为此，我们提出专为低精度前向传播设计的量化零阶优化（QuZO）框架。该方法避免了易出错的低精度直通估计器，采用优化的随机舍入缓解偏差增加问题。实验表明，QuZO在${\rm FP}8$下性能可媲美一阶方法，在${\rm INT}8$和${\rm INT}4$训练中表现更优。在LLaMA2-7B微调中，相比量化一阶方法，内存开销降低$2.94 \times$，且在GLUE、多选与生成任务上达到与MeZO相当的性能。
原文摘要 · Abstract (English)
Language Models (LLMs) are often quantized to lower precision to reduce the memory cost and latency in inference. However, quantization often degrades model performance, thus fine-tuning is required for various down-stream tasks. Traditional fine-tuning methods such as stochastic gradient descent and Adam optimization require backpropagation, which are error-prone in the low-precision settings. To overcome these limitations, we propose the Quantized Zeroth-Order (QuZO) framework, specifically designed for fine-tuning LLMs through low-precision (e.g., 4- or 8-bit) forward passes. Our method can avoid the error-prone low-precision straight-through estimator, and utilizes optimized stochastic rounding to mitigate the increased bias. QuZO simplifies the training process, while achieving results comparable to first-order methods in ${\rm FP}8$ and superior accuracy in ${\rm INT}8$ and ${\rm INT}4$ training. Experiments demonstrate that low-bit training QuZO achieves performance comparable to MeZO optimization on GLUE, Multi-Choice, and Generation tasks, while reducing memory cost by $2.94 \times$ in LLaMA2-7B fine-tuning compared to quantized first-order methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。