揭示LoRA微调在非线性条件下收敛的梯度机制，突破传统理论局限。
Gradient dynamics for low-rank fine-tuning beyond kernels
- 在师生框架下分析低秩微调，通过在线梯度下降实现收敛
- 证明在秩1扰动与原权重同量级时，收敛需dk^O(1)次迭代
- 首次摆脱对激活函数赫尔米特展开的依赖，适合研究模型优化者
LoRA已成为低成本微调大模型的主流方法，仅训练预训练模型权重的低秩扰动。尽管实证成功，其学习机制仍缺乏数学理解。本文在师生设置下研究低秩微调：给定两层基础模型 $f$ 及独立同分布样本 $(x,f^*(x))$，其中 $x$ 为高斯输入，$f^*$ 是通过秩-1矩阵扰动 $f$ 权重得到的教师模型。该设定推广了广义线性模型（GLM）回归（此时 $f$ 权重为零）。当秩-1扰动范数与 $f$ 权重矩阵相当，训练动态呈现非线性特征。在弱假设下，我们证明从基础模型初始化的学生模型，经在线梯度下降训练后，可在 $dk^{O(1)}$ 次迭代内收敛至教师模型，其中 $k$ 为 $f$ 的神经元数量。重要的是，与GLM情形不同，收敛复杂度不依赖激活函数的赫尔米特展开精细性质。此外还证明，从零开始学习教师模型所需迭代次数显著更多。
原文摘要 · Abstract (English)
LoRA has emerged as one of the de facto methods for fine-tuning foundation models with low computational cost and memory footprint. The idea is to only train a low-rank perturbation to the weights of a pre-trained model, given supervised data for a downstream task. Despite its empirical sucess, from a mathematical perspective it remains poorly understood what learning mechanisms ensure that gradient descent converges to useful low-rank perturbations. In this work we study low-rank fine-tuning in a student-teacher setting. We are given the weights of a two-layer base model $f$, as well as i.i.d. samples $(x,f^*(x))$ where $x$ is Gaussian and $f^*$ is the teacher model given by perturbing the weights of $f$ by a rank-1 matrix. This generalizes the setting of generalized linear model (GLM) regression where the weights of $f$ are zero. When the rank-1 perturbation is comparable in norm to the weight matrix of $f$, the training dynamics are nonlinear. Nevertheless, in this regime we prove under mild assumptions that a student model which is initialized at the base model and trained with online gradient descent will converge to the teacher in $dk^{O(1)}$ iterations, where $k$ is the number of neurons in $f$. Importantly, unlike in the GLM setting, the complexity does not depend on fine-grained properties of the activation's Hermite expansion. We also prove that in our setting, learning the teacher model "from scratch'' can require significantly more iterations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。