用帕累托前沿评估认证训练,发现以往方法普遍欠调优。
Rethinking Evaluation Paradigms in IBP-based Certified Training

- 通过多目标优化找最优自然与认证准确率平衡点
- 新方法发现旧配置性能被低估,提升整体表现
- 首次全面对比各类方法,揭示互补性而非单纯领先
深度神经网络在监督学习任务中表现优异,但对对抗扰动仍脆弱。神经网络验证可提供严格的鲁棒性保证,但计算成本高昂。为缓解此问题,认证训练技术在训练过程中优化可验证的鲁棒性,通常在自然准确率与认证准确率之间形成权衡,由特定超参数控制。由于这两项指标本质冲突,仅报告单一配置会导致结论误导,妨碍对当前技术水平的客观评估。本文提出基于帕累托前沿的评估范式,在自然-认证准确率权衡曲线上比较不同方法。为实现公平、方法无关的比较,采用高效的自动化多目标超参数优化,为每种方法识别一组帕累托最优配置。该方法常揭示先前报告配置存在显著欠调优,带来更优性能并确立新基准。基于这些前沿,首次完成认证训练方法的全面多目标对比,表明以往进展不如预期显著,并发现此前未报告的性能互补性。
原文摘要 · Abstract (English)
Deep neural networks achieve strong performance on many supervised learning tasks but remain vulnerable to adversarial perturbations. Neural network verification provides mathematically rigorous robustness guarantees, yet at substantial computational cost. To mitigate this, certified training techniques optimise for verifiable robustness during training, typically inducing a trade-off between natural and certified accuracy controlled by method-specific hyperparameters. Because these metrics are inherently conflicting, the common practice of reporting a single configuration is problematic: it can mislead conclusions about overall performance and prevents unbiased assessments of the state of the art. We address this by evaluating certified training methods via Pareto front comparisons over the natural--certified accuracy trade-off. To enable fair, method-agnostic comparisons, we perform efficient automated multi-objective hyperparameter optimisation to identify a set of Pareto-optimal configurations for each method. This approach often uncovers substantial undertuning in previously reported configurations, yielding superior performance and establishing a new state of the art. Leveraging these fronts, we present the first comprehensive multi-objective comparison of certified training approaches, showing that prior advancements are less pronounced than assumed and revealing previously unreported performance complementarities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。