提出统一评估大模型安全检测器的新方法,避免过拟合与参数泄露。
Gate AI: LLM Security Benchmark Evaluation Methodology and Results

- 用16个公开数据集、5折交叉验证,统一全局阈值避免逐数据集调参。
- 在12,111个样本上测试,最大F1下控制假阳性率≤1%,提升泛化能力。
- 新增多项诊断工具,可识别数据泄露、长度偏差等系统性问题,适合安全研究者参考。
大型语言模型提示注入与越狱检测器的现有评估常存在两大系统性缺陷:基于数据集的阈值调优和操作点不透明。本文提出一种评估框架,通过16个公共基准(共12,111个样本)进行5折交叉验证来解决此问题。主要采用按行分层的StratifiedKFold;同时并行运行基于复合键(父提示ID+MinHash+LSH近似重复聚类,杰卡德相似度≥0.8)的StratifiedGroupKFold,作为防泄露诊断。在保留折中选取单一全局操作点(最大F1且假阳性率≤1%),并统一应用于所有数据集,使结果反映单一阈值而非各数据集优化。通过一系列诊断测试(留一数据集交叉验证、随机标签控制、对抗验证、置换特征重要性、长度偏倚相关性、分类头一致性、跨源近似重复检测、阈值可迁移性、训练集与外部集一致性、重述不变性探测)检验泛化性能,多数设有定量通过阈值,其余标明失效模式。所有外部对比均将被测检测器阈值重新调至对手公布的误报率,确保在相同操作点下公平比较。
原文摘要 · Abstract (English)
Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored across 16 public benchmarks (12,111 samples) using 5-fold cross-validation. StratifiedKFold (by row) is the headline pass; a parallel StratifiedGroupKFold pass over a composite key (parent-prompt id plus MinHash + LSH near-duplicate clusters at Jaccard $\gtrsim 0.8$) runs alongside it as a leakage-premium diagnostic. A single global operating point is selected on the held-out folds (max F1 subject to FPR $\leq 1\%$) and applied uniformly to every dataset, so per-dataset results reflect one threshold rather than per-benchmark optimisation. Generalisation is examined through a battery of diagnostics (leave-one-dataset-out cross-validation, a random-label control, adversarial validation, permutation feature importance, length-bias correlation, classifier-head agreement, cross-source near-duplicate detection, threshold transferability, train-vs-OOF agreement, and a paraphrase-invariance probe), most with a quantitative pass threshold and the remainder with a stated failure mode. For every external comparison, the detector's threshold is re-tuned to the competitor's published false-positive rate so head-to-head values are evaluated at matched operating points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。