arXiv:2510.25130cs.LGcs.AI2025-10

通过改进非线性激活函数的线性拼接,更精准地计算模型鲁棒性边界。

Lipschitz-aware Linearity Grafting for Certified Robustness

  • 将线性部分嫁接到关键非线性激活函数上,减少近似误差
  • 在不依赖训练的情况下,使l∞局部Lipschitz常数更紧致
  • 理论证明该方法可显著提升对抗鲁棒性认证效果

Lipschitz常数是认证鲁棒性的基础,其值越小,模型在置信预测下的抗对抗样本能力越强。然而,寻找最坏情况对抗样本已被证明是NP完全问题。尽管过逼近方法在神经网络验证中取得成功,但降低逼近误差仍是重大挑战。这些误差阻碍了紧致局部Lipschitz常数的获取,而后者对认证鲁棒性至关重要。此前研究提出将线性性嫁接到非线性激活函数以减少不稳定神经元数量,实现可扩展的完整验证。但尚无理论解释其如何提升认证鲁棒性。本文认为,线性嫁接的核心作用在于消除逼近误差而非减少不稳定神经元,因为线性函数无需松弛。我们提出两个理论贡献:1)从l∞局部Lipschitz常数视角解释线性嫁接为何提升认证鲁棒性;2)将线性嫁接到主要来源的非线性激活函数上,能获得更紧致的局部Lipschitz常数。基于此,我们提出一种感知Lipschitz的线性嫁接方法,有效消除主导逼近误差,从而提升认证鲁棒性,即使无认证训练亦然。大量实验表明,对关键激活函数进行线性嫁接可收紧l∞局部Lipschitz常数并增强认证鲁棒性。

原文摘要 · Abstract (English)

Lipschitz constant is a fundamental property in certified robustness, as smaller values imply robustness to adversarial examples when a model is confident in its prediction. However, identifying the worst-case adversarial examples is known to be an NP-complete problem. Although over-approximation methods have shown success in neural network verification to address this challenge, reducing approximation errors remains a significant obstacle. Furthermore, these approximation errors hinder the ability to obtain tight local Lipschitz constants, which are crucial for certified robustness. Originally, grafting linearity into non-linear activation functions was proposed to reduce the number of unstable neurons, enabling scalable and complete verification. However, no prior theoretical analysis has explained how linearity grafting improves certified robustness. We instead consider linearity grafting primarily as a means of eliminating approximation errors rather than reducing the number of unstable neurons, since linear functions do not require relaxation. In this paper, we provide two theoretical contributions: 1) why linearity grafting improves certified robustness through the lens of the $l_\infty$ local Lipschitz constant, and 2) grafting linearity into non-linear activation functions, the dominant source of approximation errors, yields a tighter local Lipschitz constant. Based on these theoretical contributions, we propose a Lipschitz-aware linearity grafting method that removes dominant approximation errors, which are crucial for tightening the local Lipschitz constant, thereby improving certified robustness, even without certified training. Our extensive experiments demonstrate that grafting linearity into these influential activations tightens the $l_\infty$ local Lipschitz constant and enhances certified robustness.

认证鲁棒性Lipschitz常数对抗攻击神经网络验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。