arXiv:2410.06287cs.LGcs.AI2024-10被引 4

发现大模型生成永不停止的漏洞,仅用简单提示就能让主流模型陷入无限循环。

Non-Halting Queries: Exploiting Fixed Points in LLMs

  • 利用自回归模型的固定点特性,设计永远不输出结束符的提示
  • 在温度为0时证明循环令牌序列会导致模型永不终止
  • 对主流模型有效,3个词提示即可以15%成功率引发无限循环

我们揭示了一种新型漏洞,利用自回归模型中的固定点构造永不终止的查询。具体而言,非终止查询使大模型始终不采样结束符<eos>。在温度为零时,我们严格证明:若输出中出现超过上下文长度的重复(循环)令牌序列,则模型不会终止。实验表明,在基础未对齐模型中,重复提示会立即引发预测的循环行为。我们进一步提出一个简单方法,将基础模型中观察到的固定点转化为可攻击对齐模型的提示结构。该方法成功使过去一年发布的所有主流模型陷入非终止状态,即使在较高温度下也有效。在100个随机令牌的实验中,成功率从GPT-4o的97%到Gemini Pro 1.5的19%不等。结果表明,该对抗性方法绕过对齐机制的成功率比早期报告高出一到两个数量级。我们还使用ARCA进行基于梯度的直接反演,从10,000个随机2周期输出中反演出三词提示,其中1,512个成功诱导非终止状态,成功率15%。实验显示,仅需3个输入词即可高概率诱发非终止行为。总体表明,非终止查询普遍存在且易于触发。

原文摘要 · Abstract (English)

We introduce a new vulnerability that exploits fixed points in autoregressive models and use it to craft queries that never halt. More precisely, for non-halting queries, the LLM never samples the end-of-string token <eos>. We rigorously analyze the conditions under which the non-halting anomaly presents itself. In particular, at temperature zero, we prove that if a repeating (cyclic) token sequence is observed at the output beyond the context size, then the LLM does not halt. We demonstrate non-halting queries in many experiments performed in base unaligned models where repeating prompts immediately lead to a non-halting cyclic behavior as predicted by the analysis. Further, we develop a simple recipe that takes the same fixed points observed in the base model and creates a prompt structure to target aligned models. We demonstrate the recipe's success in sending every major model released over the past year into a non-halting state with the same simple prompt even over higher temperatures. Further, we devise an experiment with 100 randomly selected tokens and show that the recipe to create non-halting queries succeeds with high success rates ranging from 97% for GPT-4o to 19% for Gemini Pro 1.5. These results show that the proposed adversarial recipe succeeds in bypassing alignment at one to two orders of magnitude higher rates compared to earlier reports. We also study gradient-based direct inversion using ARCA to craft new short prompts to induce the non-halting state. We inverted 10,000 random repeating 2-cycle outputs for llama-3.1-8b-instruct. Out of 10,000 three-token inverted prompts 1,512 yield non-halting queries reaching a rate of 15%. Our experiments with ARCA show that non-halting may be easily induced with as few as 3 input tokens with high probability. Overall, our experiments demonstrate that non-halting queries are prevalent and relatively easy to find.

模型安全无限循环对抗攻击大模型漏洞

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。