arXiv:2608.09624cs.CLcs.AI2026-08

现有安全评分会误判越危险的提示反而越安全,导致攻击成功却逃过检测。

Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks

论文配图:Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks
图 1 · 摘自论文原文
  • 用固定坐标探测注意力分布,避免提示内容变化干扰评分
  • 包装后提示生成危害内容概率从5%升至27%,但评分显示更安全
  • 多种攻击和模型验证下,成功攻击反而被判定为低风险

内部安全评分在文本生成前评估提示,通常依据其能否区分有害与良性提示。然而,有害意图是提示属性,而越狱成功是目标模型、解码策略和评判者共同作用的结果。基于注意力的评分依赖于提示内容位置,若提示被包装,既改变内容又改变信号位置。本文提出主动注意力探测(Active Attention Probing),采用内容无关的固定测量坐标。将每个基础目标配对生成普通与包装版本,并在Llama模型上生成实际输出。结果显示,包装后有害生成率从0.05升至0.27,有害意图的AUROC从0.936降至0.803,即攻击更危险但评分显示更安全。在包装后的有害提示中,成功率的AUROC仅为0.220,表明成功攻击反而被判定为更低风险。罕见词、被动式及检测器衍生通道均重现此反向现象,且该现象在三个目标模型、七类攻击方法和两个独立评判者间保持一致。分布偏移先破坏校准性与阈值迁移,再影响排序性能。

原文摘要 · Abstract (English)

Internal safety scores judge a prompt before any text is generated, and they are validated by how well they separate harmful prompts from benign ones. That separation is then read as evidence that the score will also catch the attacks that succeed. Harmful intent is a property of the prompt. Jailbreak success is an outcome produced later by a particular target model, decoding policy, and judge. A filter tuned on a score that measures the wrong quantity spends its false positive budget on attacks that would have failed anyway. In this paper we audit that inference. Attention based measurements are usually read from prompt dependent locations, so a wrapper changes both the content being judged and the place the signal is taken from. We therefore introduce Active Attention Probing, which supplies a fixed content independent measurement coordinate. We pair every base goal with a plain and a wrapped version and generate real completions from the target models. On Llama, wrapping raises harmful generation from 0.05 to 0.27 while harmful intent AUROC falls from 0.936 to 0.803, so the attacks grow more dangerous while the prompts look safer to the score. Among wrapped harmful prompts the outcome AUROC is 0.220, which places the attacks that succeeded below the attacks that failed. Rare token, passive, and detector derived channels reproduce the reversal on the same matched design, and the reversal itself persists across three target models, seven attack families, and two independent judges. Distribution shift then degrades calibration and threshold transfer before it degrades ranking.

安全评估越狱攻击模型检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。