自动确定LoRA适配器的最佳放置位置，提升大模型微调效果。
PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
- 基于理论分析，自动识别适合加LoRA的模型模块
- 在监督微调和推理强化学习中表现优于现有策略
- 无需手动调参，适合希望高效微调大模型的研究者
低秩适应（LoRA）是一种广泛使用的大型模型微调方法，其小内存占用使得在极低成本下将大模型适配到特定任务成为可能。已有研究通过调整学习率、秩和初始化等方式提升效率，另一方向是适配器放置策略：实践中通常选择在注意力模块（如Query和Key）中添加LoRA。然而，关于最佳放置位置的研究较少，结论不一：原始论文建议在注意力模块放置，其他工作则推荐在MLP模块。本文通过直观的理论分析，提出PLoP（Precise LoRA Placement），一种轻量级方法，可在给定预训练模型和微调任务时，自动确定应放置LoRA适配器的模块类型。我们在监督微调和推理强化学习任务上进行了全面实验，结果表明PLoP始终优于或至少等同于常用放置策略。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) is a widely used finetuning method for large models. Its small memory footprint allows practitioners to adapt large models to specific tasks at a fraction of the cost of full finetuning. Different modifications have been proposed to enhance its efficiency by, for example, setting the learning rate, the rank, and the initialization. Another improvement axis is adapter placement strategy: when using LoRA, practitioners usually pick module types to adapt with LoRA, such as Query and Key modules. Few works have studied the problem of adapter placement, with nonconclusive results: original LoRA paper suggested placing adapters in attention modules, while other works suggested placing them in the MLP modules. Through an intuitive theoretical analysis, we introduce PLoP (Precise LoRA Placement), a lightweight method that allows automatic identification of module types where LoRA adapters should be placed, given a pretrained model and a finetuning task. We demonstrate that PLoP consistently outperforms, and in the worst case competes, with commonly used placement strategies through comprehensive experiments on supervised finetuning and reinforcement learning for reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。