通过交替优化提升大模型联邦微调的鲁棒性与表达能力
Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA
- 采用交替优化策略同时学习LoRA的上下投影矩阵
- 在多个语言模型和联邦场景下实现更优性能与收敛性
- 适合追求高效、稳定联邦微调的从业者参考
参数高效微调（PEFT）方法如低秩适应（LoRA）可通过降低计算和通信开销来优化联邦训练。我们提出RoLoRA，一种基于交替优化的联邦微调框架，用于更新LoRA适配器。该方法强调同时学习上投影和下投影矩阵对提升模型表达力与鲁棒性的关键作用。我们通过理论分析和大量实验，证明了RoLoRA优于以往仅生成不完整模型更新或限制表达力的方法。我们在线性模型上进行理论分析，突出学习上下投影矩阵的重要性；并在非线性模型上验证该洞察，并在一般条件下提供收敛性证明。为连接理论与实践，我们在包括RoBERTa-Large和Llama-2-7B在内的多种语言模型上，于多样任务与联邦学习设置下进行了广泛评估，充分展示了RoLoRA的优势。
原文摘要 · Abstract (English)
Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) optimize federated training by reducing computational and communication costs. We propose RoLoRA, a federated framework using alternating optimization to fine-tune LoRA adapters. Our approach emphasizes the importance of learning up and down projection matrices to enhance expressiveness and robustness. We use both theoretical analysis and extensive experiments to demonstrate the advantages of RoLoRA over prior approaches that either generate imperfect model updates or limit expressiveness of the model. We provide a theoretical analysis on a linear model to highlight the importance of learning both the down-projection and up-projection matrices in LoRA. We validate the insights on a non-linear model and separately provide a convergence proof under general conditions. To bridge theory and practice, we conducted extensive experimental evaluations on language models including RoBERTa-Large, Llama-2-7B on diverse tasks and FL settings to demonstrate the advantages of RoLoRA over other methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。