基于理论改进微调方法，一步到位实现高效大模型适配
LoRA-One: One-Step Full Gradient Could Suffice for Fine-Tuning Large Language Models, Provably and Efficiently
- 用一步全梯度初始化适配器，直接对齐最优子空间
- 实测在理解、推理、代码生成任务上优于原有方法
- 适合追求高效微调的AI研发人员参考
本文以大语言模型的低秩适配（LoRA）为例，探索理论如何指导实际算法设计。我们严格证明，在梯度下降下，LoRA适配器会与一步全微调梯度的特定奇异子空间对齐。这一发现表明，若用一步全梯度正确初始化适配器，可立即实现子空间对齐，且适用于线性和非线性模型。基于此理论，我们提出理论驱动的LoRA-One算法，其具备线性收敛性及良好泛化能力，引入预条件器理论上可缓解病态问题。此外，理论揭示了LoRA-One与其他梯度对齐方法的关联，澄清了相关算法设计中的误解。在自然语言理解、数学推理和代码生成等多个基准测试中，LoRA-One显著优于LoRA及其变体。代码已开源：https://github.com/YuanheZ/LoRA-One。
原文摘要 · Abstract (English)
This paper explores how theory can guide and enhance practical algorithms, using Low-Rank Adaptation (LoRA, Hu et al. 2022) in large language models as a case study. We rigorously prove that, under gradient descent, LoRA adapters align with specific singular subspaces of the one-step full fine-tuning gradient. This result suggests that, by properly initializing the adapters using the one-step full gradient, subspace alignment can be achieved immediately and applicable to both linear and nonlinear models. Building on our theory, we propose a theory-driven algorithm, LoRA-One, where the linear convergence (as well as generalization) is built and incorporating preconditioners theoretically helps mitigate the effects of ill-conditioning. Besides, our theory reveals connections between LoRA-One and other gradient-alignment-based methods, helping to clarify misconceptions in the design of such algorithms. LoRA-One achieves significant empirical improvements over LoRA and its variants across benchmarks in natural language understanding, mathematical reasoning, and code generation. Code is available at: https://github.com/YuanheZ/LoRA-One.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。