攻击大模型路由系统，让所有请求都走高价模型而不影响输出质量。
Rerouting LLM Routers
- 设计无需特定查询的干扰符号串，强制路由到高成本模型。
- 攻击在多种开源和商用路由系统上成功，且不降低生成质量。
- 低困惑度特性使基于困惑度的过滤无效，适合安全研究者关注。
大模型路由系统通过分类查询并决定调用廉价或昂贵的大模型来平衡生成质量与成本。这类系统属于我们称之为大模型控制平面的范畴——即协调多个大模型使用的系统。本文首次将大模型控制平面的完整性（即对对抗输入的鲁棒性）定义为人工智能安全中的独立问题。我们证明，攻击者可生成与查询无关的词元序列（称作'混淆装置'），将其附加到任意查询后，即可迫使大模型路由系统将该查询导向强模型。定量评估显示，该攻击在白盒与黑盒环境下均有效，针对多种开源及商业路由器均成立，且混淆查询不影响大模型输出质量。此外，这些装置可保持较低困惑度，因此基于困惑度的过滤无法作为有效防御。最后，我们探讨了其他潜在防御方案。
原文摘要 · Abstract (English)
LLM routers aim to balance quality and cost of generation by classifying queries and routing them to a cheaper or more expensive LLM depending on their complexity. Routers represent one type of what we call LLM control planes: systems that orchestrate use of one or more LLMs. In this paper, we investigate routers' adversarial robustness. We first define LLM control plane integrity, i.e., robustness of LLM orchestration to adversarial inputs, as a distinct problem in AI safety. Next, we demonstrate that an adversary can generate query-independent token sequences we call ``confounder gadgets'' that, when added to any query, cause LLM routers to send the query to a strong LLM. Our quantitative evaluation shows that this attack is successful both in white-box and black-box settings against a variety of open-source and commercial routers, and that confounding queries do not affect the quality of LLM responses. Finally, we demonstrate that gadgets can be effective while maintaining low perplexity, thus perplexity-based filtering is not an effective defense. We finish by investigating alternative defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。