用评估模型检测恶意提示，有效防御随机改写攻击。
Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation
- 通过反复调用评估模型检测提示中的危险行为，识别潜在越狱
- 对原始论文中100%的越狱攻击和99.8%的复现攻击实现拦截
- 可适配小型模型，适合快速部署到各类生成式AI系统
近期研究显示，通过重复使用随机改写（如大小写、标点等）的Best-of-N（BoN）越狱方法能有效攻破所有主流大语言模型。我们发现，使用防御暗提示（DATDP）方法后，原始论文中100%的越狱成功案例（置信区间[99.65%, 100.00%]）及我们复现中99.8%的成功越狱（置信区间[99.28%, 99.98%]）均被阻断。DATDP算法通过持续调用评估型LLM对提示进行危险性或操纵性评估——不同于其他方法，它还显式识别越狱尝试——直到生成稳健的安全评分。该效果在使用较小模型（如Claude和LLaMa-3-8B-instruct）作为评估器时依然成立，表明即使输入看似无害，语言模型仍具备识别其潜在风险的能力。因此，DATDP可低成本集成至生成式AI系统，显著提升安全性。
原文摘要 · Abstract (English)
Recent work showed Best-of-N (BoN) jailbreaking using repeated use of random augmentations (such as capitalization, punctuation, etc) is effective against all major large language models (LLMs). We have found that $100\%$ of the BoN paper's successful jailbreaks (confidence interval $[99.65\%, 100.00\%]$) and $99.8\%$ of successful jailbreaks in our replication (confidence interval $[99.28\%, 99.98\%]$) were blocked with our Defense Against The Dark Prompts (DATDP) method. The DATDP algorithm works by repeatedly utilizing an evaluation LLM to evaluate a prompt for dangerous or manipulative behaviors--unlike some other approaches, DATDP also explicitly looks for jailbreaking attempts--until a robust safety rating is generated. This success persisted even when utilizing smaller LLMs to power the evaluation (Claude and LLaMa-3-8B-instruct proved almost equally capable). These results show that, though language models are sensitive to seemingly innocuous changes to inputs, they seem also capable of successfully evaluating the dangers of these inputs. Versions of DATDP can therefore be added cheaply to generative AI systems to produce an immediate significant increase in safety.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。