用指数梯度法优化攻击后缀,实现对大模型的高效越狱。
Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent
- 直接优化松弛的一热编码,避免离散搜索与嵌入投影。
- 在5个开源模型上成功率超基线,收敛速度更快。
- 生成通用攻击后缀,可跨模型迁移,适合安全测试者使用。
随着大型语言模型(LLMs)在关键应用中日益普及,其鲁棒性与安全对齐仍是重大挑战。尽管强化学习人类反馈(RLHF)在常规提示上表现良好,但模型仍易受通过用户提示附加恶意触发词引发的越狱攻击。现有方法或依赖低效的离散标记空间搜索,或直接优化连续嵌入,后者虽可输入部分开源模型,却无法用于专有模型。将嵌入投影回有效离散标记会增加复杂性并降低攻击效果。本文提出一种内在优化方法,采用指数梯度下降结合Bregman投影,直接优化对抗后缀标记的松弛一热编码,确保每标记的优化结果始终位于概率单纯形内。我们提供了该方法的收敛性理论证明,并实现了高效算法,在五个开源LLM和四个越狱评估数据集上,相比三种先进基线,显著提升成功率并加速收敛。此外,我们还生成了对多提示有效的通用对抗后缀,并验证了优化后缀在不同模型间的可迁移性。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly deployed in critical applications, ensuring their robustness and safety alignment remains a major challenge. Despite the overall success of alignment techniques such as reinforcement learning from human feedback (RLHF) on typical prompts, LLMs remain vulnerable to jailbreak attacks enabled by crafted adversarial triggers appended to user prompts. Most existing jailbreak methods either rely on inefficient searches over discrete token spaces or direct optimization of continuous embeddings. While continuous embeddings can be given directly to selected open-source models as input, doing so is not feasible for proprietary models. On the other hand, projecting these embeddings back into valid discrete tokens introduces additional complexity and often reduces attack effectiveness. We propose an intrinsic optimization method which directly optimizes relaxed one-hot encodings of the adversarial suffix tokens using exponentiated gradient descent coupled with Bregman projection, ensuring that the optimized one-hot encoding of each token always remains within the probability simplex. We provide theoretical proof of convergence for our proposed method and implement an efficient algorithm that effectively jailbreaks several widely used LLMs. Our method achieves higher success rates and faster convergence compared to three state-of-the-art baselines, evaluated on five open-source LLMs and four adversarial behavior datasets curated for evaluating jailbreak methods. In addition to individual prompt attacks, we also generate universal adversarial suffixes effective across multiple prompts and demonstrate transferability of optimized suffixes to different LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。