让大模型微调更省内存，自动学出高效扰动策略。
Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
- 用学习方法自动优化扰动策略，取代人工固定采样
- 在4个模型7个数据集上82.1%组合表现优于旧方法
- 一次训练可复用多任务，适合频繁微调的场景
零阶优化器因无需反向传播，显著降低大语言模型（LLMs）微调时的内存开销，成为新兴方案。然而现有方法依赖静态、手工设计的采样策略，难以适配不同模型结构。为此，本文提出 ZO-Finetuner，一种基于学习的零阶优化器，通过紧凑且低内存的设计，自动学习高效的扰动策略。鉴于少量基础模型被反复用于各类下游任务，该方法支持对每个模型进行一次性的优化器训练，并在后续任务中以极小开销复用。因此，针对特定大模型仅需一次训练即可实现跨任务通用，具备高可行性和实用性。实验在4个大模型和7个数据集上验证，82.1%的任务-模型组合性能超越现有零阶基线，展现出强大的性能与可扩展性。代码已开源。
原文摘要 · Abstract (English)
Zeroth-order optimizers have recently emerged as an attractive approach for fine-tuning large language models (LLMs), as they avoid backpropagation and can substantially reduce memory overhead relative to standard first-order training. However, existing zeroth-order methods rely on hand-crafted, static sampling strategies that are not adaptable to model-specific structures. To address this, we propose ZO-Finetuner, a learning-based zeroth-order optimizer for LLMs that automatically learns efficient perturbation strategies through a compact and memory-efficient design. Motivated by the fact that a small set of base LLMs is repeatedly fine-tuned across tasks, ZO-Finetuner supports one-time per-model training and reuse across downstream tasks with minimal overhead. Therefore, learning the optimizer once for a given LLM and reusing it across diverse downstream tasks is both feasible and highly desirable. Accordingly, ZO-Finetuner is designed to scale learning to learn (L2L) to the foundation-model era by supporting one-time per-model training with minimal overhead. Experiments on 4 LLMs and 7 datasets show that ZO-Finetuner outperforms prior zeroth-order baselines in 82.1\% of task-model combinations, thereby demonstrating strong performance and scalability for efficient LLM fine-tuning. The code can be found in https://github.com/ASTRAL-Group/ZO_Fine_tuner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。