用历史记录提升大模型越狱效率,减少尝试次数
TrailBlazer: History-Guided Reinforcement Learning for Black-Box LLM Jailbreaking
- 基于历史交互信息重加权漏洞信号,指导后续攻击
- 在AdvBench和HarmBench上成功率领先,查询次数更少
- 适合研究大模型安全与对抗攻击的学者参考
大型语言模型(LLMs)已广泛应用于多个领域,其安全性成为关键问题。现有越狱方法包括提示优化、自动化红队测试、混淆及基于强化学习(RL)的方法,但大多未能有效利用早期交互中暴露的漏洞,导致攻击效率低且不稳定。由于越狱是序列化交互过程,每一步响应影响后续行为,强化学习为此提供了天然框架。本文提出一种历史感知的强化学习越狱框架,通过分析并重加权先前步骤中的漏洞信号来指导未来决策。实验表明,仅引入历史信息即可提升越狱成功率。在此基础上,我们设计了基于注意力的重加权机制,突出交互历史中的关键漏洞,实现更高效的探索,显著减少查询次数。在AdvBench和HarmBench上的大量实验显示,该方法达到当前最优越狱性能,同时大幅提升查询效率。结果强调了历史漏洞信号在强化学习驱动的越狱策略中的重要性,并为推进大模型防护的对抗研究提供了系统性路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have become integral to many domains, making their safety a critical priority. Prior jailbreaking research has explored diverse approaches, including prompt optimization, automated red teaming, obfuscation, and reinforcement learning (RL) based methods. However, most existing techniques fail to effectively leverage vulnerabilities revealed in earlier interaction turns, resulting in inefficient and unstable attacks. Since jailbreaking involves sequential interactions in which each response influences future actions, reinforcement learning provides a natural framework for this problem. Motivated by this, we propose a history-aware RL-based jailbreak framework that analyzes and reweights vulnerability signals from prior steps to guide future decisions. We show that incorporating historical information alone improves jailbreak success rates. Building on this insight, we introduce an attention-based reweighting mechanism that highlights critical vulnerabilities within the interaction history, enabling more efficient exploration with fewer queries. Extensive experiments on AdvBench and HarmBench demonstrate that our method achieves state-of-the-art jailbreak performance while significantly improving query efficiency. These results underscore the importance of historical vulnerability signals in reinforcement learning-driven jailbreak strategies and offer a principled pathway for advancing adversarial research on LLM safeguards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。