提升深度神经网络的可信鲁棒性与准确率平衡
Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifier
- 基于概率贝叶斯框架，构建带认证鲁棒半径的平滑多数投票分类器
- 理论证明在扰动半径内泛化误差界依然成立，且可推广至任意输入扰动
- 提出低成本谱正则化方法，显著增强模型鲁棒性，适合安全关键场景
在概率贝叶斯框架下，吉布斯分类器（基于后验分布 $Q$）及其加权多数投票分类器常用于分析泛化性能。然而，关于多数投票分类器的认证鲁棒性及其与泛化关系的理论研究仍较匮乏。本文提出了一个通用的泛化误差界，该界对平滑多数投票分类器（即对平滑输入的 $Q$-加权多数投票分类器）具有认证鲁棒半径——意味着在任意不超过该半径的输入扰动下，泛化界依然成立。作为副产品，我们发现泛化界与认证鲁棒半径均部分依赖于权重的谱范数，这启发我们在平滑训练中引入谱正则化以提升认证鲁棒性。利用球面高斯输入在平滑训练中的维度无关特性，我们设计了一种新颖且低成本的谱正则化器来增强平滑多数投票分类器。除了理论贡献外，还提供了实证结果验证所提方法的有效性。
原文摘要 · Abstract (English)
Within the PAC-Bayesian framework, the Gibbs classifier (defined on a posterior $Q$) and the corresponding $Q$-weighted majority vote classifier are commonly used to analyze the generalization performance. However, there exists a notable lack in theoretical research exploring the certified robustness of majority vote classifier and its interplay with generalization. In this study, we develop a generalization error bound that possesses a certified robust radius for the smoothed majority vote classifier (i.e., the $Q$-weighted majority vote classifier with smoothed inputs); In other words, the generalization bound holds under any data perturbation within the certified robust radius. As a byproduct, we find that the underpinnings of both the generalization bound and the certified robust radius draw, in part, upon weight spectral norm, which thereby inspires the adoption of spectral regularization in smooth training to boost certified robustness. Utilizing the dimension-independent property of spherical Gaussian inputs in smooth training, we propose a novel and inexpensive spectral regularizer to enhance the smoothed majority vote classifier. In addition to the theoretical contribution, a set of empirical results is provided to substantiate the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。