通过耗散层设计,实现卷积神经网络的严格鲁棒性保障。
LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layers
- 用二维罗塞尔模型直接参数化卷积核,保证每层满足李普希茨约束。
- 训练后无需额外计算开销,推理速度比傅里叶域方法快数个数量级。
- 支持多种标准卷积层结构,适用于机器人、自动驾驶等实时系统。
我们提出一种新型的卷积神经网络(CNN)逐层参数化方法,通过强制设定李普希茨边界,内置鲁棒性保障。每一层均设计为满足线性矩阵不等式(LMI),进而保证对特定供给率的耗散性。这些逐层LMI共同确保了网络输入输出映射的李普希茨有界性,其表达能力优于谱范数或正交层。新方法LipKernel利用二维罗塞尔型状态空间模型直接参数化耗散卷积核,训练后保持标准形式,评估无计算开销。数值实验表明,该方法运行时间比现有最先进的傅里叶域参数化方法快数个数量级,特别适合提升机器人、自动驾驶或自动化系统中学习型实时感知与控制的鲁棒性。本方法聚焦于CNN,兼容1维和2维卷积、最大/平均池化、步幅与空洞卷积及零填充等多种常用层;同时可自然扩展至任意增量耗散层。
原文摘要 · Abstract (English)
We propose a novel layer-wise parameterization for convolutional neural networks (CNNs) that includes built-in robustness guarantees by enforcing a prescribed Lipschitz bound. Each layer in our parameterization is designed to satisfy a linear matrix inequality (LMI), which in turn implies dissipativity with respect to a specific supply rate. Collectively, these layer-wise LMIs ensure Lipschitz boundedness for the input-output mapping of the neural network, yielding a more expressive parameterization than through spectral bounds or orthogonal layers. Our new method LipKernel directly parameterizes dissipative convolution kernels using a 2-D Roesser-type state space model. This means that the convolutional layers are given in standard form after training and can be evaluated without computational overhead. In numerical experiments, we show that the run-time using our method is orders of magnitude faster than state-of-the-art Lipschitz-bounded networks that parameterize convolutions in the Fourier domain, making our approach particularly attractive for improving the robustness of learning-based real-time perception or control in robotics, autonomous vehicles, or automation systems. We focus on CNNs, and in contrast to previous works, our approach accommodates a wide variety of layers typically used in CNNs, including 1-D and 2-D convolutional layers, maximum and average pooling layers, as well as strided and dilated convolutions and zero padding. However, our approach naturally extends beyond CNNs as we can incorporate any layer that is incrementally dissipative.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。