通过定向采样加速零阶微调，提升大模型训练效率
ConMeZO: Adaptive Descent-Direction Sampling for Gradient-Free Finetuning of Large Language Models
- 在动量方向附近构建锥形区域，自适应采样下降方向
- 实测微调速度比MeZO快2倍，保持低内存占用
- 适合资源受限下需快速微调大模型的场景
零阶优化（MeZO）因无需反向传播，可避免大语言模型微调中的内存开销，但其在千亿参数高维空间中搜索下降方向时面临维度灾难，收敛缓慢。本文提出ConMeZO，一种新型零阶优化器，通过将方向采样限制在以动量估计为中心的锥形区域内，使搜索集中在真实梯度更可能出现的方向，从而缓解高维影响。理论证明ConMeZO与MeZO具有相同的最坏情况收敛速率。实验表明，在自然语言任务上微调大模型时，ConMeZO相比MeZO最高提速2倍，同时维持零阶方法的低内存特性。
原文摘要 · Abstract (English)
Zeroth-order or derivative-free optimization (MeZO) is an attractive strategy for finetuning large language models (LLMs) because it eliminates the memory overhead of backpropagation. However, it converges slowly due to the inherent curse of dimensionality when searching for descent directions in the high-dimensional parameter space of billion-scale LLMs. We propose ConMeZO, a novel zeroth-order optimizer that accelerates convergence by adaptive directional sampling. Instead of drawing the direction uniformly at random, ConMeZO restricts the sampling to a cone centered around a momentum estimate. This concentrates the search in directions where the true gradient is more likely to lie and thus reduces the effect of high dimensions. We prove that ConMeZO achieves the same worst-case convergence rate as MeZO. Empirically, when finetuning LLMs on natural language tasks, ConMeZO is up to 2X faster than MeZO while retaining the low-memory footprint of zeroth-order methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。