arXiv:2604.27487cs.LGcs.CR2026-04

发现对抗扰动有低秩特性,提升黑盒攻击效率。

Low Rank Adaptation for Adversarial Perturbation

论文配图:Low Rank Adaptation for Adversarial Perturbation
图 1 · 摘自论文原文
  • 用参考模型将梯度投影到低维子空间
  • 在低秩子空间中搜索扰动,减少查询次数
  • 适用于高效黑盒攻击,适合安全测试场景

低秩适应(LoRA)通过低秩矩阵更新神经网络层,显著提升了大语言模型的训练效率。由于对抗样本生成是一个类似模型训练的优化过程,我们自然提出疑问:对抗扰动是否也具有类似的低秩结构?本文通过理论分析与大量实证研究,覆盖多种攻击方法、模型架构和数据集,证实对抗扰动确实具备内在的低秩结构。这一发现为提升对抗攻击与防御提供了新思路。我们主要利用该低秩特性改进黑盒攻击的效率与效果,这类攻击常因查询开销过大而受限。方法分为两步:首先,使用参考模型和辅助数据将梯度投影至低维子空间;其次,将黑盒攻击中的扰动搜索限制在此低秩子空间内,显著提升攻击效率与有效性。我们在多种攻击方法、基准模型、数据集及威胁模型下评估,结果表明,相比传统方法,本方法在性能上实现显著且一致的提升。

原文摘要 · Abstract (English)

Low-Rank Adaptation (LoRA), which leverages the insight that model updates typically reside in a low-dimensional space, has significantly improved the training efficiency of Large Language Models (LLMs) by updating neural network layers using low-rank matrices. Since the generation of adversarial examples is an optimization process analogous to model training, this naturally raises the question: Do adversarial perturbations exhibit a similar low-rank structure? In this paper, we provide both theoretical analysis and extensive empirical investigation across various attack methods, model architectures, and datasets to show that adversarial perturbations indeed possess an inherently low-rank structure. This insight opens up new opportunities for improving both adversarial attacks and defenses. We mainly focus on leveraging this low-rank property to improve the efficiency and effectiveness of black-box adversarial attacks, which often suffer from excessive query requirements. Our method follows a two-step approach. First, we use a reference model and auxiliary data to guide the projection of gradients into a low-dimensional subspace. Next, we confine the perturbation search in black-box attacks to this low-rank subspace, significantly improving the efficiency and effectiveness of the adversarial attacks. We evaluated our approach across a range of attack methods, benchmark models, datasets, and threat models. The results demonstrate substantial and consistent improvements in the performance of our low-rank adversarial attacks compared to conventional methods.

对抗攻击低秩黑盒攻击效率提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。