提出可证明Lipschitz约束的ResNet设计方法,提升模型鲁棒性。
L-Lipschitz Gershgorin ResNet Network
- 用LMI框架重构ResNet,推导参数闭式约束保证Lipschitz连续
- 基于Gershgorin定理近似特征值,确保矩阵负半定性
- 适用于对抗鲁棒性、可证明训练等场景,适合安全关键系统
深度残差网络(ResNets)在计算机视觉任务中表现卓越,得益于其保持深层梯度流动的能力。同时,控制神经网络的Lipschitz界已成为提升对抗鲁棒性和网络可验证性的关键研究方向。本文采用严格方法,基于线性矩阵不等式(LMI)框架设计$l$-Lipschitz深度残差网络。将ResNet架构重构为带有非对角元素的伪三对角LMI,推导出保证$l$-Lipschitz连续性的参数闭式约束。针对此类矩阵结构缺乏显式特征值计算的问题,采用Gershgorin圆定理近似特征值位置,确保LMI负半定性。主要贡献包括可证明的参数化方法和用于管理分层架构中递归系统的组合框架。这些成果使鲁棒网络设计成为可能,适用于对抗鲁棒性、可证明训练及控制系统。然而,发现基于Gershgorin的近似存在过约束问题,抑制了非线性动态,降低了网络表达能力。
原文摘要 · Abstract (English)
Deep residual networks (ResNets) have demonstrated outstanding success in computer vision tasks, attributed to their ability to maintain gradient flow through deep architectures. Simultaneously, controlling the Lipschitz bound in neural networks has emerged as an essential area of research for enhancing adversarial robustness and network certifiability. This paper uses a rigorous approach to design $\mathcal{L}$-Lipschitz deep residual networks using a Linear Matrix Inequality (LMI) framework. The ResNet architecture was reformulated as a pseudo-tri-diagonal LMI with off-diagonal elements and derived closed-form constraints on network parameters to ensure $\mathcal{L}$-Lipschitz continuity. To address the lack of explicit eigenvalue computations for such matrix structures, the Gershgorin circle theorem was employed to approximate eigenvalue locations, guaranteeing the LMI's negative semi-definiteness. Our contributions include a provable parameterization methodology for constructing Lipschitz-constrained networks and a compositional framework for managing recursive systems within hierarchical architectures. These findings enable robust network designs applicable to adversarial robustness, certified training, and control systems. However, a limitation was identified in the Gershgorin-based approximations, which over-constrain the system, suppressing non-linear dynamics and diminishing the network's expressive capacity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。