通过拼接LoRA实现多技能组合，显著提升复杂任务表现。
LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
- 用拼接法合并独立训练的LoRA模块，实现技能融合。
- 在数学应用题上比现有方法平均提升43%和12%。
- 适合无标注数据时快速构建复合能力模型，如专业问答系统。
低秩适应（LoRA）是高效微调大语言模型的常用技术。本文研究如何将多个LoRA模块合并，以实现技能组合——即在目标任务中整合来自单个LoRA的多种技能。该场景适用于目标任务缺乏训练数据但可分解为多个独立技能的情形。我们识别出实际应用场景，如用代码解决复杂数学文字题、基于专有手册创建问答机器人等。主要贡献在于证明拼接（CAT）方式能最优加权各技能对应的LoRA，性能优于现有模型与数据融合方法：例如在数学应用题上，平均领先43%和12%。这是首个表明模型合并优于数据混合的二元技能组合研究。代码与数据已开源。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs). We study how different LoRA modules can be merged to achieve skill composition -- testing the performance of the merged model on a target task that involves combining multiple skills, each skill coming from a single LoRA. This setup is favorable when it is difficult to obtain training data for the target task and when it can be decomposed into multiple skills. First, we identify practically occurring use-cases that can be studied under the realm of skill composition, e.g. solving hard math-word problems with code, creating a bot to answer questions on proprietary manuals or about domain-specialized corpora. Our main contribution is to show that concatenation of LoRAs (CAT), which optimally weights LoRAs that were individually trained on different skills, outperforms existing model- and data- merging techniques; for instance on math-word problems, CAT beats these methods by an average of 43% and 12% respectively. Thus, this paper advocates model merging as an efficient way to solve compositional tasks and underscores CAT as a simple, compute-friendly and effective procedure. To our knowledge, this is the first work demonstrating the superiority of model merging over data mixing for binary skill composition tasks. Code and data are available at https://github.com/aksh555/LoRA-Soups
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。