提出低秩零阶优化算法，高效微调大模型且内存开销极小。
Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures
- 用低秩结构近似梯度，避免存储激活值。
- 在多种模型和任务上性能接近一阶方法。
- 兼容动量法，内存增加可忽略。
参数高效微调（PEFT）显著降低了大语言模型（LLMs）在下游任务中适配的内存开销。然而，传统一阶（FO）微调算法因需存储反向传播中的激活值，导致内存占用较高，尤其在长上下文微调任务中更为突出。零阶（ZO）算法通过有限差分近似函数值来估算梯度，无需存储激活值，提供了有前景的替代方案。但现有ZO方法难以捕捉LLM微调中常见的低秩梯度结构，导致性能不佳。本文提出一种低秩零阶梯度估计器，并引入新型低秩零阶算法（LOZO），有效捕获该结构。我们将LOZO建模为子空间优化方法，提供收敛性保证。其低秩特性使LOZO能与动量技术结合，额外内存开销可忽略。大量实验表明，LOZO及其动量变体在多种模型规模和下游任务中均优于现有ZO方法，且性能接近一阶算法。
原文摘要 · Abstract (English)
Parameter-efficient fine-tuning (PEFT) significantly reduces memory costs when adapting large language models (LLMs) for downstream applications. However, traditional first-order (FO) fine-tuning algorithms incur substantial memory overhead due to the need to store activation values for back-propagation during gradient computation, particularly in long-context fine-tuning tasks. Zeroth-order (ZO) algorithms offer a promising alternative by approximating gradients using finite differences of function values, thus eliminating the need for activation storage. Nevertheless, existing ZO methods struggle to capture the low-rank gradient structure common in LLM fine-tuning, leading to suboptimal performance. This paper proposes a low-rank ZO gradient estimator and introduces a novel low-rank ZO algorithm (LOZO) that effectively captures this structure in LLMs. We provide convergence guarantees for LOZO by framing it as a subspace optimization method. Additionally, its low-rank nature enables LOZO to integrate with momentum techniques while incurring negligible extra memory costs. Extensive experiments across various model sizes and downstream tasks demonstrate that LOZO and its momentum-based variant outperform existing ZO methods and closely approach the performance of FO algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。