同一模型不同随机种子导致认证鲁棒性差异极大,影响安全应用可信度。
On the Extreme Variance of Certified Local Robustness Across Model Seeds
- 通过对比不同随机种子训练的模型,发现认证鲁棒性波动剧烈。
- 标准差超过近期论文报告的鲁棒性提升量,表明结果不可靠。
- 适合关注模型可靠性与验证全面性的研究人员参考。
神经网络的鲁棒性验证对于安全关键应用至关重要,但其结果可能受机器学习中随机性的影响。尽管准确性受随机性影响已有广泛研究,其对认证鲁棒性验证的影响仍未知。本文揭示了一个令人担忧的现象:仅因训练时随机种子不同,模型的认证鲁棒性表现出极端差异,标准差甚至超过近期论文报告的边际改进量。此外,认证鲁棒性在未见数据上的泛化能力也显著变化,难以满足安全任务的可靠性要求。这一发现表明:(i) 认证鲁棒性结果因极端波动而缺乏说服力;(ii) 测试集中的‘幸运’模型种子无法保证在新测试集上保持高鲁棒性。为此,我们呼吁研究者报告认证鲁棒性的置信区间,并建议验证时采用大规模、多样化且未见的数据进行更全面评估。
原文摘要 · Abstract (English)
Robustness verification of neural networks, referring to formally proving that neural networks satisfy robustness properties, is of crucial importance in safety-critical applications, where model failures can result in loss of human life or million-dollar damages. However, the dependability of verification results may be questioned due to sources of randomness in machine learning, and although this has been widely investigated for accuracy, its impact on robustness verification remains unknown. In this paper, we demonstrate a concerning result: Models that differ only in random seeds during training exhibit extreme variance in their certified robustness, with a standard deviation that is statistically larger than the marginal robustness improvements reported in recent machine learning papers. In addition, we also show that certified robustness generalization to unseen data varies significantly across datasets, falling short of the dependability expectations for safety-critical tasks. Our findings are major concerns because: (i) machine learning results in certified robustness are likely unconvincing due to extreme variance in certified robustness, and (ii) a ``lucky'' model seed in a test set cannot be guaranteed to maintain its higher certified robustness under a different test set. In light of these results, we urge researchers to increase the reporting of confidence intervals for certified robustness, and we urge those verifying neural networks to be more comprehensive in verification by using large-scale, diverse, and unseen data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。