arXiv:2608.06674cs.CVcs.LG2026-08

首次针对检测模型注意力机制发起隐形攻击,显著降低检测精度。

Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformers

论文配图:Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformers
图 1 · 摘自论文原文
  • 直接优化编码器注意力目标,用不可察觉的扰动破坏模型空间推理。
  • 在COCO上使DETR-R50 mAP从42.1降至0.97,比现有最强攻击下降超4倍。
  • 对多种注意力结构均有效,适合研究模型鲁棒性与安全性的学者。

对抗漏洞仍是神经网络安全部署的重大挑战,尤其在物体检测这一嵌入众多关键系统的核心任务中。检测用的变压器模型虽已成为主流,但其对抗鲁棒性仍相对未被充分探索。现有攻击多聚焦于检测输出,而非模型特有的注意力机制。本文提出首个在不可察觉、有界ℓ∞扰动下,直接优化编码器注意力目标的攻击方法。不同于通过可见补丁引入攻击者拥有的汇点,本方法引导模型自身注意力向受污染目标集中。我们认为编码器注意力集中了模型的空间推理能力,因此破坏它比仅扰动检测输出更具破坏性。实验显示,该攻击使DETR-R50在COCO上的mAP从42.1降至0.97,在相同扰动预算和迭代次数下,相比最强现有攻击,结果mAP下降约4倍。进一步验证表明,此脆弱性不依赖特定污染目标:在四种不同目标(分散、重排序、置换、峰值抑制)下,检测性能始终低于3 mAP,说明弱点源于注意力结构本身被破坏,而非单一目标。最后,攻击在多种注意力形式间泛化良好:使DINO-Swin-L的mAP从56.8降至1.44,优于之前最强攻击的7.3,确立了密集与可变形注意力下的新基准。

原文摘要 · Abstract (English)

Adversarial vulnerabilities remain a major concern for the safe deployment of neural networks, particularly in object detection, a core task embedded in many safety-critical systems. Detection transformers have emerged as leading object detectors, yet their adversarial robustness remains comparatively underexplored. Most existing attacks target the detection output rather than the attention mechanism that makes these models distinctive. In this paper, we introduce the first attack that directly optimizes an encoder-attention objective under an imperceptible, bounded $\ell_\infty$ perturbation. Rather than introducing an attacker-owned sink token through a visible patch, it drives the model's own attention toward a corrupted target. We argue that encoder attention concentrates the model's spatial reasoning, so corrupting it propagates through the detection pipeline more disruptively than perturbing the detection output alone. Our attack reduces DETR-R50 mAP on COCO from 42.1 to 0.97, a $\sim 4\times$ reduction in resulting mAP over the strongest existing attack under an identical perturbation budget and iteration count. We further show that this vulnerability is not specific to a particular corruption objective: across four qualitatively distinct targets, dispersion, re-ranking, permutation, and peak-suppression, detection consistently drops below 3 mAP, suggesting that the weakness arises from disrupting the attention structure itself rather than from any single target. Finally, we demonstrate that the attack generalizes across attention formulations, reducing DINO-Swin-L from 56.8 to 1.44 mAP against 7.3 for the strongest prior attack, establishing state-of-the-art on both dense and deformable attention.

对抗攻击检测模型注意力机制鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。