提出可计算的 Lipschitz 上界，提升神经网络鲁棒性验证精度。
Computable Lipschitz Bounds for Deep Neural Networks
- 基于 l¹ 与 l∞ 范数设计新上界，适用于全连接与卷积网络。
- 在四个测试中均优于现有方法，首个解析测试上界精确。
- 适合关注模型鲁棒性验证的研究者使用。
推导深度神经网络 Lipschitz 常数的紧致且可计算的上界，对形式化保证神经网络模型的鲁棒性至关重要。本文分析了针对 l² 范数的三种现有上界，强调 l¹ 与 l∞ 范数的重要性，并为前馈全连接网络与卷积神经网络提出了两个新上界。针对卷积网络的技术难点，采用显式与隐式两种方法处理。通过四项数值实验验证理论结果：两个基于解析闭式输出的测试、一个随机矩阵测试，以及在 MNIST 数据集上训练的卷积网络测试。结果显示，其中一个新上界在最简解析测试中为精确值，其余测试中也优于其他上界。
原文摘要 · Abstract (English)
Deriving sharp and computable upper bounds of the Lipschitz constant of deep neural networks is crucial to formally guarantee the robustness of neural-network based models. We analyse three existing upper bounds written for the $l^2$ norm. We highlight the importance of working with the $l^1$ and $l^\infty$ norms and we propose two novel bounds for both feed-forward fully-connected neural networks and convolutional neural networks. We treat the technical difficulties related to convolutional neural networks with two different methods, called explicit and implicit. Several numerical tests empirically confirm the theoretical results, help to quantify the relationship between the presented bounds and establish the better accuracy of the new bounds. Four numerical tests are studied: two where the output is derived from an analytical closed form are proposed; another one with random matrices; and the last one for convolutional neural networks trained on the MNIST dataset. We observe that one of our bound is optimal in the sense that it is exact for the first test with the simplest analytical form and it is better than other bounds for the other tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。