arXiv:2504.18598cs.CRcs.AI2025-04被引 16

攻击者通过激活闲置专家实现对MoE模型的隐蔽控制。

BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts

  • 识别未使用专家并优化触发词以激活它们。
  • 在保持模型整体性能前提下成功植入恶意指令。
  • 适合研究模型安全与后门防御的从业者参考。

Mixture-of-Experts(MoE)作为大语言模型的高效架构,通过将不同输入分发至特定专家网络实现能力扩展与计算成本的平衡。然而,该类模型的安全漏洞尚未被充分研究,后门攻击仍属空白。本文首次提出针对MoE模型的后门攻击方法BadMoE:攻击者通过优化路由触发词,激活原本未被使用的“闲置专家”,并将其提升为决定模型输出的“主导专家”。我们证明了少数主导专家的存在,并发现闲置专家可被操控以改变预测结果。具体而言,攻击流程包括:1)筛选与目标任务无关的闲置专家;2)设计路由感知损失函数,优化其激活触发词;3)通过污染训练数据使这些专家获得主导地位。大量实验表明,BadMoE能在不损害模型整体性能的前提下,精准执行攻击者预设的恶意任务,相较于现有方法更具隐蔽性与有效性。

原文摘要 · Abstract (English)

Mixture-of-Experts (MoE) have emerged as a powerful architecture for large language models (LLMs), enabling efficient scaling of model capacity while maintaining manageable computational costs. The key advantage lies in their ability to route different tokens to different ``expert'' networks within the model, enabling specialization and efficient handling of diverse input. However, the vulnerabilities of MoE-based LLMs still have barely been studied, and the potential for backdoor attacks in this context remains largely unexplored. This paper presents the first backdoor attack against MoE-based LLMs where the attackers poison ``dormant experts'' (i.e., underutilized experts) and activate them by optimizing routing triggers, thereby gaining control over the model's output. We first rigorously prove the existence of a few ``dominating experts'' in MoE models, whose outputs can determine the overall MoE's output. We also show that dormant experts can serve as dominating experts to manipulate model predictions. Accordingly, our attack, namely BadMoE, exploits the unique architecture of MoE models by 1) identifying dormant experts unrelated to the target task, 2) constructing a routing-aware loss to optimize the activation triggers of these experts, and 3) promoting dormant experts to dominating roles via poisoned training data. Extensive experiments show that BadMoE successfully enforces malicious prediction on attackers' target tasks while preserving overall model utility, making it a more potent and stealthy attack than existing methods.

后门攻击MoE模型安全防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。