探索联邦大模型的可行性与安全挑战，提出未来研究方向
Federated Large Language Models: Feasibility, Robustness, Security and Future Directions
- 从可行性和鲁棒性角度系统分析联邦大模型技术
- 揭示数据异构和隐私泄露等核心风险
- 适合关注隐私计算与大模型安全的研究者
大型语言模型（LLM）与联邦学习（FL）的融合为分布式数据联合训练提供了隐私保护方案，解决了数据孤岛问题。然而，这一新兴领域——联邦大语言模型（FLLM）——面临通信与计算开销、异构性、隐私与安全等挑战。现有研究多集中于可行性，未来趋势将聚焦系统鲁棒性与安全性提升。本文全面综述了FLLM的最新进展，从可行性、鲁棒性、安全性及未来方向四方面展开：系统梳理了现有可行性研究，提出了应对资源、数据与任务异构性的鲁棒性增强方法，分析了隐私威胁与安全风险，回顾了防御机制，并探讨了少样本学习、机器遗忘与知识产权保护等前沿方向。研究强调亟需进一步探索以提升系统鲁棒性与安全性。
原文摘要 · Abstract (English)
The integration of Large Language Models (LLMs) and Federated Learning (FL) presents a promising solution for joint training on distributed data while preserving privacy and addressing data silo issues. However, this emerging field, known as Federated Large Language Models (FLLM), faces significant challenges, including communication and computation overheads, heterogeneity, privacy and security concerns. Current research has primarily focused on the feasibility of FLLM, but future trends are expected to emphasize enhancing system robustness and security. This paper provides a comprehensive review of the latest advancements in FLLM, examining challenges from four critical perspectives: feasibility, robustness, security, and future directions. We present an exhaustive survey of existing studies on FLLM feasibility, introduce methods to enhance robustness in the face of resource, data, and task heterogeneity, and analyze novel risks associated with this integration, including privacy threats and security challenges. We also review the latest developments in defense mechanisms and explore promising future research directions, such as few-shot learning, machine unlearning, and IP protection. This survey highlights the pressing need for further research to enhance system robustness and security while addressing the unique challenges posed by the integration of FL and LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。