arXiv:2508.08040cs.LGcs.AI2025-08

提出首个针对多模态联邦提示学习的后门攻击,隐蔽性强且成功率超90%

BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Models

  • 恶意客户端协同优化触发器与提示向量,污染全局提示
  • 攻击成功率>90%,仅需少量参与客户端即可生效
  • 适用于研究联邦学习安全性的研究人员,警示实际部署风险

基于提示的微调已成为大视觉语言模型中轻量级替代全参数微调的方法,通过学习上下文提示实现高效适配。该范式最近被扩展至联邦学习场景(如PromptFL),在数据隐私约束下各客户端协作训练提示。然而,提示聚合在联邦多模态学习中的安全影响仍基本未被探索,存在关键攻击面。本文提出\textbf{BadPromptFL},首个针对多模态对比模型中基于提示的联邦学习的后门攻击。在BadPromptFL中,被攻陷的客户端联合优化本地后门触发器与提示嵌入,将中毒提示注入全局聚合过程。这些提示随后传播至良性客户端,在推理时无需修改模型参数即可实现通用后门激活。借助CLIP类架构的上下文学习特性,BadPromptFL在少量客户端参与下实现>90%的攻击成功率,具有极强隐蔽性。在多个数据集和聚合协议上的实验验证了攻击的有效性、隐蔽性与泛化能力,凸显了提示型联邦学习在真实部署中的鲁棒性隐患。

原文摘要 · Abstract (English)

Prompt-based tuning has emerged as a lightweight alternative to full fine-tuning in large vision-language models, enabling efficient adaptation via learned contextual prompts. This paradigm has recently been extended to federated learning settings (e.g., PromptFL), where clients collaboratively train prompts under data privacy constraints. However, the security implications of prompt-based aggregation in federated multimodal learning remain largely unexplored, leaving a critical attack surface unaddressed. In this paper, we introduce \textbf{BadPromptFL}, the first backdoor attack targeting prompt-based federated learning in multimodal contrastive models. In BadPromptFL, compromised clients jointly optimize local backdoor triggers and prompt embeddings, injecting poisoned prompts into the global aggregation process. These prompts are then propagated to benign clients, enabling universal backdoor activation at inference without modifying model parameters. Leveraging the contextual learning behavior of CLIP-style architectures, BadPromptFL achieves high attack success rates (e.g., \(>90\%\)) with minimal visibility and limited client participation. Extensive experiments across multiple datasets and aggregation protocols validate the effectiveness, stealth, and generalizability of our attack, raising critical concerns about the robustness of prompt-based federated learning in real-world deployments.

联邦学习后门攻击多模态提示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。