提出联邦提示学习统一框架,解决大模型训练中的隐私与效率难题
Federated Prompt Learning: A Unified Framework, Empirical Analysis, and Future Directions

- 构建联邦提示学习统一框架,整合提示调优与联邦学习
- 揭示提示学习在通信开销、个性化和异构性处理上的优势与权衡
- 面向隐私敏感场景的AI研发人员,提供技术选型与未来方向参考
大语言模型(LLMs)已成为学术界和工业界云智能服务的核心,但其训练与部署面临高计算成本、数据集中化及隐私问题。联邦学习(FL)提供了一种去中心化训练范式,使客户端无需共享原始数据即可协同训练模型,是实现隐私保护的大型语言模型训练与推理的有力方案。本文对联邦提示学习(FPL)进行全面综述,系统梳理了将联邦学习范式与大语言模型融合的最新进展,回答三个核心问题:RQ1——FPL的基本动机、特征与使能技术,及其与传统联邦学习和全模型微调的区别;RQ2——FPL方法在性能、通信效率、计算开销、可扩展性、个性化及异构性处理方面的权衡;RQ3——现存的安全、隐私、鲁棒性与系统挑战,以及关键未来研究方向。我们从预训练、微调到实际应用全生命周期系统分析现有FPL方法,讨论安全、隐私与鲁棒性问题,并总结现有防御机制。最后,指出开放挑战与未来方向,助力读者理解研究洞见如何推动FPL发展。
原文摘要 · Abstract (English)
Large language models (LLMs) have become core components of cloud-based intelligent services in academia and industry, yet their training and deployment are hindered by high computational costs, data centralization, and privacy concerns. Federated learning (FL) offers a decentralized training paradigm that enables clients to collaboratively train a learning model without sharing raw data, making it a promising solution for privacy-preserving LLM training and reasoning. This paper presents a comprehensive survey of federated prompt learning (FPL) to review recent advances in integrating the federated learning paradigm and large language models, answering the following research questions: RQ1: The fundamental motivations, characteristics, and enabling technologies of FPL, and how it differs from conventional FL and full-model federated fine-tuning; RQ2: The trade-offs FPL approaches exhibit in performance, communication efficiency, computational overhead, scalability, personalization, and heterogeneity handling; RQ3: The remaining security, privacy, robustness, and system challenges, along with key future research directions. To this end, we systematically examine existing FPL methods across the full model lifecycle: pre-training, fine-tuning, and practical applications, while discussing security, privacy, and robustness issues and summarizing existing defense mechanisms. Finally, we highlight open challenges and future directions, aiming to help readers understand how the insights drive research in FPL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。