混合零阶与一阶优化，高效提升大模型微调性能
Hi-ZFO: Hierarchical Zeroth- and First-Order LLM Fine-Tuning via Importance-Guided Tensor Selection
- 按层重要性划分模型，关键层用精确梯度更新
- 在生成任务中训练速度提升显著，性能优于纯一阶方法
- 适合追求高效微调的大模型应用开发者
标准一阶（FO）优化常导致大语言模型训练陷入尖锐、泛化差的极小值；而零阶（ZO）方法虽具更强探索性，但收敛慢且在生成任务中因输出空间巨大导致估计方差高，噪声大效率低。为此，我们提出Hi-ZFO（分层零阶-一阶优化），通过层级重要性分析自适应划分模型：对关键层使用精确的FO更新，对次要层引入具有‘有益随机性’的ZO优化，以帮助模型跳出一阶优化易滞留的局部极小值。在多种生成、数学及代码推理任务上验证，Hi-ZFO在显著缩短训练时间的同时持续实现更优性能，证明了分层混合优化在大模型微调中的有效性。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) using standard first-order (FO) optimization often drives training toward sharp, poorly generalizing minima. Conversely, zeroth-order (ZO) methods offer stronger exploratory behavior without relying on explicit gradients, yet suffer from slow convergence. More critically, our analysis reveals that in generative tasks, the vast output and search space significantly amplify estimation variance, rendering ZO methods both noisy and inefficient. To address these challenges, we propose \textbf{Hi-ZFO} (\textbf{Hi}erarchical \textbf{Z}eroth- and \textbf{F}irst-\textbf{O}rder optimization), a hybrid framework designed to synergize the precision of FO gradients with the exploratory capability of ZO estimation. Hi-ZFO adaptively partitions the model through layer-wise importance profiling, applying precise FO updates to critical layers while leveraging ZO optimization for less sensitive ones. Notably, ZO in Hi-ZFO is not merely a memory-saving surrogate; it is intentionally introduced as a source of "beneficial stochasticity" to help the model escape the local minima where pure FO optimization tends to stagnate. Validated across diverse generative, mathematical, and code reasoning tasks, Hi-ZFO consistently achieves superior performance while significantly reducing the training time. These results demonstrate the effectiveness of hierarchical hybrid optimization for LLM fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。