arXiv:2412.12449stat.MLcs.LG2024-12被引 1

通过雅可比正则化,建立对抗鲁棒性与泛化性的理论联系。

Adversarially robust generalization theory via Jacobian regularization for deep neural networks

  • 用雅可比范数正则化,逼近对抗攻击下的损失上界。
  • 证明雅可比范数影响标准与鲁棒泛化差距,实验证明其有效性。
  • 适合关注对抗训练理论和模型泛化的研究人员阅读。

强大的深度神经网络易受对抗攻击。为获得对抗鲁棒模型,研究者分别发展了对抗训练与雅可比正则化技术。对抗训练已有大量理论与实证研究,但雅可比正则化的理论基础仍不充分。本文表明,$\\(ell_{2}$ 或 $\\(ell_{1}$ 雅可比正则化损失在 $\\(ell_{2}$ 或 $\\(ell_{\infty}$ 对抗攻击下,分别作为对抗鲁棒损失的近似上界。进一步,通过约束标准损失函数类与雅可比正则化函数类的 Rademacher 复杂度,建立了雅可比正则化风险最小化器的鲁棒泛化差距。理论结果表明,雅可比范数同时关联标准与鲁棒泛化。在 MNIST 数据分类任务上的实验表明,雅可比正则化风险最小化确实可作为对抗鲁棒风险最小化的代理,降低雅可比范数能提升标准与鲁棒泛化性能。该研究推动了基于雅可比正则化的对抗鲁棒泛化理论与实证理解。

原文摘要 · Abstract (English)

Powerful deep neural networks are vulnerable to adversarial attacks. To obtain adversarially robust models, researchers have separately developed adversarial training and Jacobian regularization techniques. There are abundant theoretical and empirical studies for adversarial training, but theoretical foundations for Jacobian regularization are still lacking. In this study, we show that Jacobian regularization is closely related to adversarial training in that $\ell_{2}$ or $\ell_{1}$ Jacobian regularized loss serves as an approximate upper bound on the adversarially robust loss under $\ell_{2}$ or $\ell_{\infty}$ adversarial attack respectively. Further, we establish the robust generalization gap for Jacobian regularized risk minimizer via bounding the Rademacher complexity of both the standard loss function class and Jacobian regularization function class. Our theoretical results indicate that the norms of Jacobian are related to both standard and robust generalization. We also perform experiments on MNIST data classification to demonstrate that Jacobian regularized risk minimization indeed serves as a surrogate for adversarially robust risk minimization, and that reducing the norms of Jacobian can improve both standard and robust generalization. This study promotes both theoretical and empirical understandings to adversarially robust generalization via Jacobian regularization.

对抗鲁棒雅可比正则泛化理论深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。