arXiv:2510.05261cs.LG2025-10中稿 · Transactions on Ma…被引 2

提出高效局部 Lipschitz 估计方法,显著提升神经网络鲁棒性验证速度与精度。

ECLipsE-Gen-Local: Efficient Compositional Local Lipschitz Estimates for Deep Neural Networks

  • 构建可分解的广义 SDP 框架,支持异质激活函数和任意子网络
  • 计算复杂度线性随深度增长,比传统方法快数倍且更紧致
  • 适用于小输入区域时逼近精确雅可比值,适合实际鲁棒性分析

Lipschitz 常数是衡量神经网络对输入扰动鲁棒性的关键指标。但精确计算该常数为 NP 难问题,现有方法需求解大规模半定规划(SDP),随网络规模急剧恶化。本文提出 ECLipsE-Gen-Local,一种高效的组合式局部 Lipschitz 估计框架。首先构建可灵活处理异质激活函数斜率、任意输入输出对及连续层子网络的广义 SDP 框架;随后将其分解为一系列小规模子问题,计算复杂度随网络深度线性增长。还设计闭式解变体,实现近实时计算。所有算法均具备理论可行性与有效性保证。实验表明,本方法在多个基准上大幅提速,同时生成比全局方法更紧的 Lipschitz 上界。当输入区域较小时,其上界接近自动微分所得精确雅可比值。进一步证明该估计能准确反映网络实际鲁棒性。

原文摘要 · Abstract (English)

The Lipschitz constant is a key measure for certifying the robustness of neural networks to input perturbations. However, computing the exact constant is NP-hard, and standard approaches to estimate the Lipschitz constant involve solving a large matrix semidefinite program (SDP) that scales poorly with network size. Further, there is a potential to efficiently leverage local information on the input region to provide tighter Lipschitz estimates. We address this problem here by proposing a compositional framework that yields tight yet scalable Lipschitz estimates for deep feedforward neural networks. Specifically, we begin by developing a generalized SDP framework that is highly flexible, accommodating heterogeneous activation function slope, and allowing Lipschitz estimates with respect to arbitrary input-output pairs and arbitrary choices of sub-networks of consecutive layers. We then decompose this generalized SDP into a sequence of small sub-problems, with computational complexity that scales linearly with respect to the network depth. We also develop a variant that achieves near-instantaneous computation through closed-form solutions to each sub-problem. All our algorithms are accompanied by theoretical guarantees on feasibility and validity. Next, we develop a series of algorithms, termed as ECLipsE-Gen-Local, that effectively incorporate local information on the input. Our experiments demonstrate that our algorithms achieve substantial speedups over a multitude of benchmarks while producing significantly tighter Lipschitz bounds than global approaches. Moreover, we show that our algorithms provide strict upper bounds for the Lipschitz constant with values approaching the exact Jacobian from autodiff when the input region is small enough. Finally, we demonstrate the practical utility of our approach by showing that our Lipschitz estimates closely align with network robustness.

鲁棒性验证Lipschitz估计深度网络高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。