用收缩理论提升卷积神经ODE的抗噪和抗攻击能力
Robust Convolution Neural ODEs via Contractivity-promoting regularization
- 通过引入雅可比矩阵正则化，使网络动态系统具备收缩性
- 在MNIST和FashionMNIST上噪声和对抗攻击下准确率提升10%以上
- 适用于激活函数有斜率限制的卷积神经ODE，计算开销低
神经网络对输入噪声和对抗攻击敏感。本文研究连续深度网络——卷积神经微分方程（Convolutional NODEs），提出利用收缩理论提升其鲁棒性。收缩系统中，不同初始条件的轨迹会指数快速收敛。具有收缩性的卷积NODEs能有效抑制特征扰动带来的输出变化。训练时通过包含系统动力学雅可比矩阵的正则项可诱导收缩性。为降低计算负担，我们证明对斜率受限激活函数的一类NODEs，仅需精心设计的权重正则化即可实现收缩性。在MNIST和FashionMNIST数据集上的图像分类任务中，使用多种噪声和攻击测试，验证了所提正则化方法的有效性。
原文摘要 · Abstract (English)
Neural networks can be fragile to input noise and adversarial attacks. In this work, we consider Convolutional Neural Ordinary Differential Equations (NODEs), a family of continuous-depth neural networks represented by dynamical systems, and propose to use contraction theory to improve their robustness. For a contractive dynamical system two trajectories starting from different initial conditions converge to each other exponentially fast. Contractive Convolutional NODEs can enjoy increased robustness as slight perturbations of the features do not cause a significant change in the output. Contractivity can be induced during training by using a regularization term involving the Jacobian of the system dynamics. To reduce the computational burden, we show that it can also be promoted using carefully selected weight regularization terms for a class of NODEs with slope-restricted activation functions. The performance of the proposed regularizers is illustrated through benchmark image classification tasks on MNIST and FashionMNIST datasets, where images are corrupted by different kinds of noise and attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。