不用训练就能检测视觉对抗补丁，靠检索相似攻击实现智能识别。
Don't Lag, RAG: Training-Free Adversarial Detection Using RAG
- 用视觉语言模型检索已存攻击样本，生成推理判断新攻击类型。
- 开源模型最高达95%准确率，闭源Gemini-2.0达98%领先水平。
- 无需标注、不需训练，适合实时部署的防御系统使用。
对抗补丁攻击通过局部扰动误导深度视觉模型，构成严重威胁。传统防御方法通常需重新训练或微调，难以在真实场景中应用。本文提出无需训练的视觉检索增强生成（VRAG）框架，利用视觉语言模型（VLMs）进行对抗补丁检测。通过在持续扩展的数据库中检索与待测补丁视觉相似的攻击样本及图像，VRAG实现生成式推理，可识别多种攻击类型，且无需额外训练或微调。我们评估了多个开源大规模VLMs，包括Qwen-VL-Plus、Qwen2.5-VL-72B和UI-TARS-72B-DPO，以及闭源模型Gemini-2.0。结果显示，开源模型UI-TARS-72B-DPO最高达到95%分类准确率，创下开源领域新纪录；Gemini-2.0整体准确率达98%，但为闭源。实验表明，VRAG能以极少人工标注有效识别各类对抗补丁，为应对不断演化的对抗补丁攻击提供了鲁棒且实用的防御方案。
原文摘要 · Abstract (English)
Adversarial patch attacks pose a major threat to vision systems by embedding localized perturbations that mislead deep models. Traditional defense methods often require retraining or fine-tuning, making them impractical for real-world deployment. We propose a training-free Visual Retrieval-Augmented Generation (VRAG) framework that integrates Vision-Language Models (VLMs) for adversarial patch detection. By retrieving visually similar patches and images that resemble stored attacks in a continuously expanding database, VRAG performs generative reasoning to identify diverse attack types, all without additional training or fine-tuning. We extensively evaluate open-source large-scale VLMs, including Qwen-VL-Plus, Qwen2.5-VL-72B, and UI-TARS-72B-DPO, alongside Gemini-2.0, a closed-source model. Notably, the open-source UI-TARS-72B-DPO model achieves up to 95 percent classification accuracy, setting a new state-of-the-art for open-source adversarial patch detection. Gemini-2.0 attains the highest overall accuracy, 98 percent, but remains closed-source. Experimental results demonstrate VRAG's effectiveness in identifying a variety of adversarial patches with minimal human annotation, paving the way for robust, practical defenses against evolving adversarial patch attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。