通过精心放置有害目标，用良性内容突破大模型安全限制。
Jailbreaking in the Haystack
- 在用户指令前插入模型生成的良性文本，隐藏有害目标。
- 在HarmBench上使多个主流模型攻击成功率大幅提升。
- 低成本、易迁移，且比增加尝试次数更高效利用算力。
近期长上下文语言模型（LMs）的发展使得百万词输入成为可能，拓展了其在复杂任务如计算机使用代理中的能力。然而，这些扩展上下文的安全影响尚不明确。为此，我们提出NINJA（针尖藏于草堆中的越狱攻击），通过在有害用户目标前附加良性、由模型生成的内容来实现对对齐模型的越狱。关键发现是：有害目标的位置在安全性中起重要作用。在标准安全基准HarmBench上的实验表明，NINJA显著提升了当前最先进的开源与专有模型（包括LLaMA、Qwen、Mistral和Gemini）的攻击成功率。与以往方法不同，该方法资源消耗低、可迁移性强且更难被检测。此外，我们证明了NINJA在计算上是最优的——在固定算力预算下，增加上下文长度的效果优于增加尝试次数的best-of-N越狱方法。这些结果揭示，即使看似无害的长上下文，若经过精心设计目标位置，也会在现代语言模型中引入根本性漏洞。
原文摘要 · Abstract (English)
Recent advances in long-context language models (LMs) have enabled million-token inputs, expanding their capabilities across complex tasks like computer-use agents. Yet, the safety implications of these extended contexts remain unclear. To bridge this gap, we introduce NINJA (short for Needle-in-haystack jailbreak attack), a method that jailbreaks aligned LMs by appending benign, model-generated content to harmful user goals. Critical to our method is the observation that the position of harmful goals play an important role in safety. Experiments on standard safety benchmark, HarmBench, show that NINJA significantly increases attack success rates across state-of-the-art open and proprietary models, including LLaMA, Qwen, Mistral, and Gemini. Unlike prior jailbreaking methods, our approach is low-resource, transferable, and less detectable. Moreover, we show that NINJA is compute-optimal -- under a fixed compute budget, increasing context length can outperform increasing the number of trials in best-of-N jailbreak. These findings reveal that even benign long contexts -- when crafted with careful goal positioning -- introduce fundamental vulnerabilities in modern LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。