arXiv:2509.21130cs.LGcs.CV2025-09

用稀疏主成分分析提升神经网络抗干扰能力,理论证明更鲁棒。

Sparse Representations Improve Adversarial Robustness of Neural Network Classifiers

  • 用稀疏主成分替代传统PCA做特征提取
  • 在强对抗攻击下性能下降更平缓,干净准确率仍高
  • 适合关注模型安全性的研究者和工程师

深度神经网络在图像分类上表现优异,但易受精心设计的对抗扰动影响。本文重新审视线性降维作为简单且数据自适应的防御方法,实证比较了标准PCA与其稀疏变体SPCA作为下游分类器前段特征提取器的效果,并辅以理论分析。理论上,推导出线性分类头应用于SPCA特征时的精确鲁棒性证书:对于ℓ∞和ℓ₂威胁模型(二分类与多分类),认证半径随投影矩阵W⊤u的对偶范数减小而增大。进一步证明,对于一般非线性分类头,稀疏性通过Lipschitz复合论证降低算子范数界,预测更低输入敏感度。实验上,使用小型非线性网络后接投影,SPCA在强白盒与黑盒攻击下始终比PCA退化更温和,同时保持竞争力的干净准确率。理论揭示机制(稀疏投影减少对抗杠杆),实验验证该优势可延续至非线性场景。代码开源于https://github.com/killian31/SPCARobustness。

原文摘要 · Abstract (English)

Deep neural networks perform remarkably well on image classification tasks but remain vulnerable to carefully crafted adversarial perturbations. This work revisits linear dimensionality reduction as a simple, data-adapted defense. We empirically compare standard Principal Component Analysis (PCA) with its sparse variant (SPCA) as front-end feature extractors for downstream classifiers, and we complement these experiments with a theoretical analysis. On the theory side, we derive exact robustness certificates for linear heads applied to SPCA features: for both $\ell_\infty$ and $\ell_2$ threat models (binary and multiclass), the certified radius grows as the dual norms of $W^\top u$ shrink, where $W$ is the projection and $u$ the head weights. We further show that for general (non-linear) heads, sparsity reduces operator-norm bounds through a Lipschitz composition argument, predicting lower input sensitivity. Empirically, with a small non-linear network after the projection, SPCA consistently degrades more gracefully than PCA under strong white-box and black-box attacks while maintaining competitive clean accuracy. Taken together, the theory identifies the mechanism (sparser projections reduce adversarial leverage) and the experiments verify that this benefit persists beyond the linear setting. Our code is available at https://github.com/killian31/SPCARobustness.

对抗鲁棒性稀疏表示主成分分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。