提出可公平评估模型各分类鲁棒性的新指标,无需攻击即可发现弱类。
GF-Score: Certified Class-Conditional Robustness Evaluation with Fairness Guarantees

- 分解鲁棒性为每类得分,用经济公平指标量化差异
- 76%的模型显示猫类最脆弱,更鲁棒模型反而类间差异更大
- 自校准机制无需对抗攻击,适合安全关键场景审计
对抗鲁棒性对安全关键应用至关重要,但现有评估方法要么需昂贵的对抗攻击,要么仅提供整体分数,掩盖了不同类别间的鲁棒性分布。我们提出 extit{GF-Score}(GREAT-Fairness Score),将认证的 GREAT 得分分解为类别级鲁棒性画像,并通过四个基于福利经济学的指标量化其差异:鲁棒性差距指数(RDI)、归一化鲁棒性吉尼系数(NRGC)、最差类别鲁棒性(WCR)以及公平惩罚型 GREAT 得分(FP-GREAT)。该框架通过仅使用干净准确率相关性的自校准过程,消除了原方法对对抗攻击的依赖。在 RobustBench 的 22 个模型上评估 CIFAR-10 与 ImageNet 数据集,结果表明分解是精确的,类别得分揭示出一致的脆弱模式(如 76\/% 的 CIFAR-10 模型中“猫”是最弱类别),且更鲁棒的模型往往表现出更大的类别级差异。这些结果建立了一个实用、免攻击的审计流程,用于诊断认证鲁棒性是否平等保护所有类别。代码已开源。
原文摘要 · Abstract (English)
Adversarial robustness is essential for deploying neural networks in safety-critical applications, yet standard evaluation methods either require expensive adversarial attacks or report only a single aggregate score that obscures how robustness is distributed across classes. We introduce the \emph{GF-Score} (GREAT-Fairness Score), a framework that decomposes the certified GREAT Score into per-class robustness profiles and quantifies their disparity through four metrics grounded in welfare economics: the Robustness Disparity Index (RDI), the Normalized Robustness Gini Coefficient (NRGC), Worst-Case Class Robustness (WCR), and a Fairness-Penalized GREAT Score (FP-GREAT). The framework further eliminates the original method's dependence on adversarial attacks through a self-calibration procedure that tunes the temperature parameter using only clean accuracy correlations. Evaluating 22 models from RobustBench across CIFAR-10 and ImageNet, we find that the decomposition is exact, that per-class scores reveal consistent vulnerability patterns (e.g., ``cat'' is the weakest class in 76\% of CIFAR-10 models), and that more robust models tend to exhibit greater class-level disparity. These results establish a practical, attack-free auditing pipeline for diagnosing where certified robustness guarantees fail to protect all classes equally. We release our code on \href{https://github.com/aryashah2k/gf-score}{GitHub}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。