将低秩微调引入联邦学习，高效保护隐私地训练大模型。
Federated Low-Rank Adaptation for Foundation Models: A Survey
- 用低秩矩阵压缩参数量，降低联邦微调的计算负担。
- 解决多设备数据异构、通信效率低等联邦学习核心难题。
- 适合关注隐私保护与资源受限场景的大模型研究者。
有效利用私有数据仍是构建基础模型的重大挑战。联邦学习（FL）作为一种协作框架，使多个用户能在不暴露数据的前提下共同微调模型，缓解数据隐私风险。与此同时，低秩适应（LoRA）通过大幅减少可训练参数，为大模型微调提供了一种资源高效的替代方案。本文综述了LoRA如何被集成到大模型的联邦微调中，该方向称为FedLoRA，重点探讨分布式学习、数据异构性和效率三大挑战，并按具体应对方法对现有工作进行分类。最后，讨论了开放的研究问题，展望未来可行的研究方向，明确推进FedLoRA的下一步路径。
原文摘要 · Abstract (English)
Effectively leveraging private datasets remains a significant challenge in developing foundation models. Federated Learning (FL) has recently emerged as a collaborative framework that enables multiple users to fine-tune these models while mitigating data privacy risks. Meanwhile, Low-Rank Adaptation (LoRA) offers a resource-efficient alternative for fine-tuning foundation models by dramatically reducing the number of trainable parameters. This survey examines how LoRA has been integrated into federated fine-tuning for foundation models, an area we term FedLoRA, by focusing on three key challenges: distributed learning, heterogeneity, and efficiency. We further categorize existing work based on the specific methods used to address each challenge. Finally, we discuss open research questions and highlight promising directions for future investigation, outlining the next steps for advancing FedLoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。