通过动态低秩压缩提升模型抗攻击能力，兼顾紧凑与鲁棒性。
Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
- 引入谱正则化控制每层低秩核心的条件数，增强稳定性。
- 在保持正常精度前提下，压缩率超94%，对抗准确率反而提升。
- 无需修改架构，适配多种模型与攻击场景，适合边缘部署。
在资源受限设备上部署神经网络需要模型既紧凑又对对抗输入具备鲁棒性。然而，压缩与对抗鲁棒性常存在矛盾。本文提出一种动态低秩训练方法，结合新型谱正则化，控制各层低秩核心的条件数。该方法在不牺牲干净数据准确率的前提下，缓解压缩模型对对抗扰动的敏感性。该方法具有模型与数据无关性，计算高效，并支持秩自适应，可自动压缩目标网络。在标准架构、数据集及多种对抗攻击下的大量实验表明，正则化网络可实现超过94%的压缩率，同时对抗准确率较未压缩基线有所恢复或提升。
原文摘要 · Abstract (English)
Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing accuracy on clean data. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。