用编码+搜索组合攻破自检防御,成功率超60%。
Borrowed Strength: Best-of-N Search over a Code EncodingBreaks Self-Check Jailbreak Defenses
- 将代码补全编码与最佳N次搜索结合,单个攻击无效但合起来可突破防御。
- 在三个开源模型上攻破率达67%/22%/15%,70B模型仍有效。
- 揭示防御机制弱点,适合安全研究者和对抗攻击方向读者。
自检防御要求目标模型在回应前评估请求;目前最强的SAGE方法平均防御成功率达99%。我们展示可通过组合两个单独无害的攻击来突破该防御:已知的代码补全编码与最佳N次搜索,二者单独使用时危害性均低于4.7%。组合后,在三个开源目标模型上分别达到67%、22%、15%的攻破率,且在70B模型上效果依然存在。我们进一步解释了这一现象:自检防御依赖目标模型自身判断,而四个目标模型将请求转化为明确拒绝的比例在32%至97%之间,导致防御覆盖范围差异显著,尽管未受保护的攻击可达性几乎相同。此外,攻击存活与否取决于防御类型,对变换类防御,代码编码保留更多未受保护能力;对门控类防御,顺序反转。我们通过攻击对防御决策边界独立探测次数进行解释。最后,我们报告并修复了自身流水线中的有效性缺陷——贪婪解码下的确定性攻击不存在最佳N次搜索的变异通道,并给出一维诊断方法。所有结论基于31万次生成结果由人工验证裁判评分支持。
原文摘要 · Abstract (English)
A self-check defense asks the target model to assess a request before answering it; SAGE, the strongest published instance, reports an average 99% defense success rate. We show it can be breached by composing two attacks that are individually harmless against it: an established code-completion encoding and an established best-of-N search, neither of which exceeds 4.7% of behaviors alone. Composed, with the search budget spent on the encoding, they reach 67/22/15% across three open targets, and the effect persists on a 70B target. We then explain the composition rather than only reporting it. First, a self-check defense borrows its strength from the target: SAGE does not detect the attack, it asks the model to, and the four targets convert that request into an explicit refusal between 32% and 97% of the time, which orders the spread in defended coverage even though undefended reach is near-identical. Second, which attack survives is decided by the type of defense, and it inverts: against transform defenses the code encoding retains far more of its undefended reach than the character search, while against gate defenses the ordering flips. We account for this with the number of independent probes an attack delivers to a defense's decision boundary. Finally, we report a validity defect we found and repaired in our own pipeline, a deterministic attack under greedy decoding has no best-of-N variation channel at all, and give the one-line diagnostic that detects it. All claims rest on 310,000 generations scored by a human-validated judge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。