混合微调新方法，兼顾效率与性能提升
New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework
- 结合零阶与一阶优化，同步更新大模型与参数高效模块
- 在多任务、多架构上均实现稳定性能提升
- 理论分析支持，适合大规模语言模型微调场景
大型语言模型（LLMs）的微调通常采用全量微调（更新所有参数）或参数高效微调（PEFT，仅调整少量参数）。前者计算开销大，后者常难以学习新知识且表现不佳。为此，我们提出一种新型混合微调方法，联合使用零阶与一阶优化策略，同步更新LLM和PEFT模块。为分析该算法，我们构建以‘混合光滑性条件’为核心的理论框架，刻画联合训练中优化景观的异质特性。推导了多学习率下重排型SGD算法的收敛性，并通过大量下游任务与模型架构的实证研究验证其有效性。实验表明，该方法在多个任务中均带来一致性能提升，为大规模语言模型微调提供了可行方案。
原文摘要 · Abstract (English)
Fine-tuning Large Language Models (LLMs) typically involves either full fine-tuning, which updates all model parameters, or Parameter-Efficient Fine-Tuning (PEFT), which adjusts a small subset of parameters. However, both approaches have inherent limitations: full fine-tuning is computationally expensive, while PEFT often struggles to learn new knowledge and exhibits suboptimal performance. To overcome these issues, we propose a novel hybrid fine-tuning approach that jointly updates both LLMs and PEFT modules using a combination of zeroth-order and first-order optimization methods. To analyze our new algorithm, we develop a theoretical framework centered on the concept of hybrid smoothness condition, which accounts for the heterogeneous nature of the optimization landscape in joint LLM and PEFT training. We derive a rigorous convergence analysis for the convergence of reshuffling-type SGD algorithm under multiple learning rates and demonstrate its effectiveness through extensive empirical studies across various downstream tasks and model architectures. On the practical side, our results demonstrate consistent performance improvement, making the approach a viable solution for large-scale language model fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。