发现MoE模型路由失衡漏洞，用简单重复文本即可让推理变慢3倍
RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress
- 通过重复令牌构造对抗性输入，诱导路由器集中调用少数专家
- 在Mixtral-8x7B上使端到端延迟增加3.063倍，严重破坏服务可用性
- 无需模型知识的黑盒攻击，适合安全测试与防御研究者参考
Mixture-of-Experts架构因参数效率高已成为大语言模型扩展的标准。为应对专家数量增长，现代推理系统普遍采用专家并行将专家分布于多个设备。然而，推理过程中缺乏显式负载均衡约束，导致恶意输入可引发严重的路由集中现象。我们证明，分布外提示能操纵路由策略，使所有令牌持续被导向同一组top-k专家，造成某些设备计算过载而其他设备闲置。这使原本高效的机制沦为拒绝服务攻击向量，违反首个响应时间的服务级别协议。本文提出轻量级黑盒攻击方法RepetitionCurse，通过识别MoE路由器行为的通用缺陷，以模型无关的方式利用简单重复的令牌模式构造对抗性提示。在广泛部署的Mixtral-8x7B等MoE模型上，该方法使端到端推理延迟提升3.063倍，显著降低服务可用性。
原文摘要 · Abstract (English)
Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, the absence of explicit load balancing constraints during inference allows adversarial inputs to trigger severe routing concentration. We demonstrate that out-of-distribution prompts can manipulate the routing strategy such that all tokens are consistently routed to the same set of top-$k$ experts, which creates computational bottlenecks on certain devices while forcing others to idle. This converts an efficiency mechanism into a denial-of-service attack vector, leading to violations of service-level agreements for time to first token. We propose RepetitionCurse, a low-cost black-box strategy to exploit this vulnerability. By identifying a universal flaw in MoE router behavior, RepetitionCurse constructs adversarial prompts using simple repetitive token patterns in a model-agnostic manner. On widely deployed MoE models like Mixtral-8x7B, our method increases end-to-end inference latency by 3.063x, degrading service availability significantly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。