自动判别器评估大模型越狱成功率不可靠，不同判别方式结果差异大且易被攻击。
How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring
- 用人类标注数据对比两类自动判别器，发现专用分类器误报多，LLM判别器召回率波动大。
- 同一内容经不同判别器评分，成功率差异可达57%至100%，部分攻击仅靠添加拒绝语句即可实现。
- 专用分类器虽抗表面攻击能力强，但白盒攻击仍能翻转70%高置信度有害样本，且危害内容未删减。
几乎所有关于大模型越狱和提示注入的研究都报告攻击成功率为ASR，而该数值由自动化判别器而非人工评定：或是针对任务训练的安全分类器，或为提示后用于评分的通用聊天模型。此类判别器极少被验证。我们使用HarmBench分类器验证集中的596个经人工标注的生成结果，对比两类判别器与人类多数意见，并对其发起攻击。专用分类器过度标记（精确率0.835，召回率0.974）；三类以LLM为判别器的系统保持较高精确率（0.81–0.94），但召回率极不稳定（0.06–0.65），相同回复在不同判别器下得分差异巨大。二者在鲁棒性上也截然不同：不修改有害文本仅添加良性框架的包装攻击，可使所有LLM判别器在57%至100%情况下翻转判断结果，其中单个前置拒绝语句导致39%至88%的翻转。专用分类器对这类表面攻击具有较强抵抗力（最高仅6.7%），但针对其开放权重的白盒GCG攻击可在小优化预算下翻转70%的高置信度真阳性样本（21/30；95% CI 54–86%），且80次采样验证均显示翻转后的文本仍含原始有害内容。由于越来越多的报告值来自LLM判别器，这些数据不仅平均不可靠，且在对抗压力下更易失效。我们建议论文报告判别器在人工标注子集上的精度与召回率，对ASR进行精度校正，并加入判别器对抗测试。代码已开源。
原文摘要 · Abstract (English)
Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model prompted to grade. The judge is rarely checked. We check it. Using 596 human-labeled completions from the HarmBench classifier validation set, we compare the two judge families against human majority votes and then attack them. The two families fail in opposite ways. The dedicated classifier over-flags (precision 0.835, recall 0.974); three different LLM-as-judges keep high precision (0.81 to 0.94) but show erratic recall (0.06 to 0.65), so the same responses produce very different ASR depending on which judge scores them. The two families also differ sharply in robustness. Wrappers that leave the harmful text untouched and only add benign framing flip every LLM-judge between 57% and 100% of the time, and a single prepended refusal sentence accounts for much of this (39% to 88%). The dedicated classifier resists these surface attacks (at most 6.7%), but a white-box GCG attack on its open weights flips 70% of confident true positives (21 of 30; 95% CI 54 to 86%) even at a small optimization budget. A two-annotator audit confirms the attacks leave the harm intact: every one of 80 sampled flips still contained the harmful content. Because a large and growing share of reported ASR comes from LLM-judges, many such numbers are unreliable both on average and under deliberate pressure. We recommend that papers report judge precision and recall on a human-labeled slice, report ASR corrected for judge precision, and include an adversarial check of the judge. Our code is released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。