arXiv:2410.11317cs.LGcs.CL2024-10被引 8

将混乱的对抗提示翻译成自然语言,提升越狱攻击成功率。

Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation

  • 将混乱对抗提示转化为可读自然语言,保留语义漏洞。
  • 10次查询内对7个闭源模型平均成功率81.8%,高于现有方法。
  • 适用于研究越狱机制或提升攻击效率的研究者。

自动对抗提示生成在越狱安全对齐的大语言模型方面表现卓越。现有基于梯度的攻击虽在白盒模型上效果显著,但常生成语义混乱的对抗提示,难以迁移至其他模型,限制了对未知目标模型的攻击能力。本文首次深入解析混乱对抗提示中的语义信息,提出一种新方法将其‘翻译’为连贯、可读的自然语言对抗提示。该方法能有效揭示触发模型漏洞的语义内容,并无损地传递至目标模型,显著增强越狱攻击效果。实验表明,本方法在最多10次查询下,于HarmBench上对7个商业闭源大模型(含GPT与Claude-3系列)实现81.8%的平均攻击成功率;在AdvBench上对抵抗性强的Llama-2-Chat模型也超过90%攻击成功率。代码已开源:https://github.com/qizhangli/Adversarial-Prompt-Translator。

原文摘要 · Abstract (English)

Automatic adversarial prompt generation provides remarkable success in jailbreaking safely-aligned large language models (LLMs). Existing gradient-based attacks, while demonstrating outstanding performance in jailbreaking white-box LLMs, often generate garbled adversarial prompts with chaotic appearance. These adversarial prompts are difficult to transfer to other LLMs, hindering their performance in attacking unknown victim models. In this paper, for the first time, we delve into the semantic meaning embedded in garbled adversarial prompts and propose a novel method that "translates" them into coherent and human-readable natural language adversarial prompts. In this way, we can effectively uncover the semantic information that triggers vulnerabilities of the model and unambiguously transfer it to the victim model, without overlooking the adversarial information hidden in the garbled text, to enhance jailbreak attacks. It also offers a new approach to discovering effective designs for jailbreak prompts, advancing the understanding of jailbreak attacks. Experimental results demonstrate that our method significantly improves the success rate of jailbreak attacks against various safety-aligned LLMs and outperforms state-of-the-arts by large margins. With at most 10 queries, our method achieves an average attack success rate of 81.8% in attacking 7 commercial closed-source LLMs, including GPT and Claude-3 series, on HarmBench. Our method also achieves over 90% attack success rates against Llama-2-Chat models on AdvBench, despite their outstanding resistance to jailbreak attacks. Code at: https://github.com/qizhangli/Adversarial-Prompt-Translator.

越狱攻击对抗提示大模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。