arXiv:2411.00985cs.CL2024-11被引 3

提出可迁移的离散提示调优方法,保护隐私并降低大模型本地部署成本。

FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Models

  • 客户端用离散提示优化,通过预测精度反馈实现无梯度调优。
  • 服务器用语义相似度与聚类筛选提示,减少通信开销。
  • 在黑盒场景下效果优于现有方法,且提示可跨任务迁移。

近年来,大语言模型(LLMs)显著推动了自然语言处理的发展。通过特定场景数据微调,基础模型可更好地适配下游任务。然而,微调过程存在隐私泄露风险,尤其在集中式数据处理中。为缓解此问题,联邦学习(FL)被引入以避免多源数据集中收集。但大模型自身的隐私同样重要,潜在恶意攻击威胁其安全,当前研究对此关注不足。因此,建立可信的多方模型微调环境至关重要。此外,本地部署大模型带来高昂存储与计算成本。为此,我们首次提出面向黑盒大语言模型的联邦离散可迁移提示调优(FedDTPT)。客户端采用基于预测精度反馈的令牌级离散提示优化,通过MLM API实现无梯度优化;服务端则利用语义相似度注意力机制过滤本地提示,并结合嵌入距离肘部检测与DBSCAN聚类提升过滤效果。实验表明,在黑盒设置下,相比现有最优方法,本方案实现更高准确率、更低通信开销,且对非独立同分布数据具有鲁棒性,优化后的提示具备可迁移性。

原文摘要 · Abstract (English)

In recent years, large language models (LLMs) have significantly advanced the field of natural language processing (NLP). By fine-tuning LLMs with data from specific scenarios, these foundation models can better adapt to various downstream tasks. However, the fine-tuning process poses privacy leakage risks, particularly in centralized data processing scenarios. To address user privacy concerns, federated learning (FL) has been introduced to mitigate the risks associated with centralized data collection from multiple sources. Nevertheless, the privacy of LLMs themselves is equally critical, as potential malicious attacks challenge their security, an issue that has received limited attention in current research. Consequently, establishing a trusted multi-party model fine-tuning environment is essential. Additionally, the local deployment of large LLMs incurs significant storage costs and high computational demands. To address these challenges, we propose for the first time a federated discrete and transferable prompt tuning, namely FedDTPT, for black-box large language models. In the client optimization phase, we adopt a token-level discrete prompt optimization method that leverages a feedback loop based on prediction accuracy to drive gradient-free prompt optimization through the MLM API. For server optimization, we employ an attention mechanism based on semantic similarity to filter all local prompt tokens, along with an embedding distance elbow detection and DBSCAN clustering strategy to enhance the filtering process. Experimental results demonstrate that, compared to state-of-the-art methods, our approach achieves higher accuracy, reduced communication overhead, and robustness to non-iid data in a black-box setting. Moreover, the optimized prompts are transferable.

联邦学习提示调优隐私保护大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。