用蜜蜂寻蜜的思路,高效找到大模型漏洞攻击路径。
Let the Bees Find the Weak Spots: A Path Planning Perspective on Multi-Turn Jailbreak Attacks against LLMs
- 将多轮攻击抽象为路径规划问题,用蜂群算法协同搜索
- 平均仅26次查询即达90%以上成功率,最高98%
- 适合安全测试人员快速评估大模型防御能力
大型语言模型广泛应用于各类场景,但其安全与伦理风险日益引发关注。现有红队测试多采用多轮越狱攻击识别模型漏洞,但常缺乏对有效对话轨迹的探索,且攻击过程开销较大。本文首次提出基于动态加权图拓扑的理论模型,将多轮攻击过程建模为路径规划问题。在此框架下,提出改进型人工蜂群算法ABC,包含雇佣蜂、观察蜂和侦察蜂的协同搜索机制。该算法显著提升最优攻击路径搜索效率,大幅降低平均查询次数。在三个开源及两个专有语言模型上的实证评估显示,本方法整体攻击成功率超90%,在GPT-3.5-Turbo上最高达98%,优于现有基线。同时,平均仅需26次查询即可达成相近效果,显著降低红队测试开销,凸显其优越效率。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have been widely deployed across various applications, yet their potential security and ethical risks have raised increasing concerns. Existing research employs red teaming evaluations, utilizing multi-turn jailbreaks to identify potential vulnerabilities in LLMs. However, these approaches often lack exploration of successful dialogue trajectories within the attack space, and they tend to overlook the considerable overhead associated with the attack process. To address these limitations, this paper first introduces a theoretical model based on dynamically weighted graph topology, abstracting the multi-turn attack process as a path planning problem. Based on this framework, we propose ABC, an enhanced Artificial Bee Colony algorithm for multi-turn jailbreaks, featuring a collaborative search mechanism with employed, onlooker, and scout bees. This algorithm significantly improves the efficiency of optimal attack path search while substantially reducing the average number of queries required. Empirical evaluations on three open-source and two proprietary language models demonstrate the effectiveness of our approach, achieving attack success rates above 90\% across the board, with a peak of 98\% on GPT-3.5-Turbo, and outperforming existing baselines. Furthermore, it achieves comparable success with only 26 queries on average, significantly reducing red teaming overhead and highlighting its superior efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。