用独立干扰块诱骗ViT注意力,让防御失效却保持攻击效果。
Adversarial Decoys: Misdirecting Attention-Based Defenses in ViT

- 设计独立干扰块,专门诱导注意力偏离真实攻击区域。
- 在ImageNet上多模型测试中,攻击成功率仍超80%且注意力被成功转移。
- 无需修改原攻击,适配任意对抗补丁攻击,可做通用防御绕过工具。
视觉变换器(ViTs)对局部对抗攻击(如对抗补丁)仍易受攻击,近期测试时防御方法通过抑制注意力得分异常高的图像标记来缓解此类攻击。这些防御依赖于注意力与攻击有效性的强关联:对抗性标记通常需吸引大量注意力才能影响预测。本文提出对抗性干扰块(adversarial decoys),即独立优化的图像块,可引导注意力及防御机制聚焦于特定目标标记。该方法将错误分类与防御规避两个目标解耦:原始攻击区域负责误导分类,而独立的干扰块则操控防御所依赖的注意力排序。采用逐层目标函数增强目标标记的注意力,并使其超越其他非目标标记。由于干扰块与原始攻击独立优化,该方法具备攻击无关性,可轻松集成至任意现有对抗补丁攻击中。在多个ViT架构和攻击场景下的ImageNet实验表明,干扰块能有效将高注意力分数从真实攻击区域转移,同时保持攻击的大部分有效性。结果揭示了以注意力强度作为对抗相关性指标的根本局限。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) remain vulnerable to localized adversarial attacks, e.g., adversarial patches, while recent test-time defenses mitigate them by suppressing image tokens with abnormally high attention scores. These defenses exploit a strong coupling between attention and adversarial effectiveness: adversarial tokens often need to attract substantial attention to influence the prediction. We introduce adversarial decoys, independently optimized image patches that redirect the attention, and therefore related defenses, toward selected target tokens. Rather than jointly optimizing misclassifications and defense evasion, our approach decouples the two objectives: the original adversarial region induces the incorrect prediction, while a separate decoy manipulates the attention ranking used by the defense. A layer-wise objective increases target-token attention and promotes these tokens above competing non-target ones. Since the decoy is optimized independently of the underlying attack, the method is attack-agnostic and can be easily integrated with any existing adversarial patch attack. Experiments on ImageNet across multiple ViT architectures and attacks show that decoys can redirect high attention scores away from the true adversarial region while preserving much of the attack effectiveness. These results reveal a fundamental limitation of using attention magnitude as an indicator of adversarial relevance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。