提出LoRA学习率与秩的最优缩放规律,实现从低秩微调到全量微调的学习率迁移。
Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning
- 基于最大更新参数化,推导出学习率随适配器秩的理论缩放规律。
- 发现两种不同缩放模式:恒定或反比于秩,取决于初始化和缩放因子。
- 验证学习率可从LoRA有效迁移到全量微调,大幅减少调参成本。
低秩适应(LoRA)是大模型参数高效微调的标准工具,虽内存开销小,但其训练动态受初始化、适配器秩和学习率等超参数影响复杂。尤其学习率如何随适配器秩变化尚不明确,导致秩调整时需重调学习率。本文提出最大更新适配(μA),一个理论框架,揭示在标准配置下,为保证稳定且不消失的特征更新,最优学习率应如何随模型宽度和适配器秩缩放。μA源自预训练中的最大更新参数化(μP)。分析结合超参数迁移技术,发现最优学习率存在两种缩放模式:一种随秩基本不变,另一种与秩成反比,具体取决于初始化和LoRA缩放因子。进一步识别出可实现从LoRA到全量微调学习率迁移的配置。跨语言、视觉、图文、图像生成及强化学习任务的实验验证了缩放规律,并表明在LoRA上调好的学习率可可靠迁移至全量微调。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) is a standard tool for parameter-efficient finetuning of large models. While it induces a small memory footprint, its training dynamics can be surprisingly complex as they depend on several hyperparameters such as initialization, adapter rank, and learning rate. In particular, it is unclear how the optimal learning rate scales with adapter rank, which forces practitioners to re-tune the learning rate whenever the rank is changed. In this paper, we introduce Maximal-Update Adaptation ($μ$A), a theoretical framework that characterizes how the "optimal" learning rate should scale with model width and adapter rank to produce stable, non-vanishing feature updates under standard configurations. $μ$A is inspired from the Maximal-Update Parametrization ($μ$P) in pretraining. Our analysis leverages techniques from hyperparameter transfer and reveals that the optimal learning rate exhibits different scaling patterns depending on initialization and LoRA scaling factor. Specifically, we identify two regimes: one where the optimal learning rate remains roughly invariant across ranks, and another where it scales inversely with rank. We further identify a configuration that allows learning rate transfer from LoRA to full finetuning, drastically reducing the cost of learning rate tuning for full finetuning. Experiments across language, vision, vision--language, image generation, and reinforcement learning tasks validate our scaling rules and show that learning rates tuned on LoRA transfer reliably to full finetuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。