联邦大模型如何在保护隐私前提下高效训练
Federated Large Language Models: Current Progress and Future Directions
- 通过联邦学习实现多客户端协作训练大模型，不共享原始数据
- 解决数据异构、通信开销大等核心挑战，提升训练稳定性
- 适合关注隐私保护与个性化模型的AI研究者和开发者
大语言模型在多个应用中表现出色，但其训练通常依赖集中式数据收集，引发严重隐私与治理问题。联邦学习提供去中心化方案，允许多个客户端协作训练共享模型而无需暴露原始本地数据。然而，将联邦学习与大语言模型结合带来新挑战，包括数据异构性、收敛不稳定、通信开销及计算资源限制。本文综述联邦大语言模型（FedLLM）的最新进展，系统梳理近期成果，重点关注联邦微调与联邦提示学习，并分析现有方法如何应对效率、个性化与安全挑战。此外，总结了联邦预训练与联邦智能体等新兴方向。旨在为这一快速发展的领域提供结构化视角，揭示未来研究的潜在路径。
原文摘要 · Abstract (English)
Large Language Models have achieved impressive performance across diverse applications, yet their training typically depends on centralized data collection, raising serious privacy and governance concerns. Federated Learning offers a decentralized alternative by enabling multiple clients to collaboratively train shared models without exposing raw local data. However, integrating FL with LLMs introduces new challenges, including data heterogeneity, convergence instability, communication overhead, and computational constraints. This survey provides a comprehensive and up-to-date overview of Federated Learning for Large Language Models (FedLLM). We systematically review recent advances, with particular emphasis on federated fine-tuning and federated prompt learning, and analyze how existing methods address efficiency, personalization, and security challenges. We further summarize emerging directions such as federated pre-training and federated agents. Our goal is to offer a structured perspective on this rapidly evolving field and to highlight promising avenues for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。