发现对抗样本在特征空间有固定偏移方向，据此设计新防御方法提升视觉语言模型鲁棒性。
Adversarial Attacks Already Tell the Answer: Directional Bias-Guided Test-time Defense for Vision-Language Models
- 利用对抗样本在特征空间的统一偏移方向，反向修正错误特征。
- 在15个数据集上实现顶尖防御效果，且对抗准确率超过干净准确率。
- 无需重新训练，适合部署在真实场景中快速响应攻击。
视觉语言模型（如CLIP）虽具备强大的零样本泛化能力，但对对抗扰动极为脆弱，威胁实际应用安全。测试时防御方法近期成为高效应对攻击的方案，无需大规模重训练。本文发现一个意外现象：在多种输入变换下，CLIP特征空间中的对抗图像始终沿一个主导方向偏移，而正常图像则分散分布。我们提出该主导偏移方向为‘防御方向’，其反向指向正确类别中心。基于此，提出方向性偏置引导防御（DBD），通过估计防御方向并采用双流重建策略恢复鲁棒表示。在15个数据集上的实验表明，DBD不仅达到当前最优的对抗鲁棒性，同时保持高干净准确率，并首次揭示对抗准确率可超越干净准确率。这说明对抗扰动本身蕴含关于真实决策边界的定向先验。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs), such as CLIP, have shown strong zero-shot generalization but remain highly vulnerable to adversarial perturbations, posing serious risks in real-world applications. Test-time defenses for VLMs have recently emerged as a promising and efficient approach to defend against adversarial attacks without requiring costly large-scale retraining. In this work, we uncover a surprising phenomenon: under diverse input transformations, adversarial images in CLIP's feature space consistently shift along a dominant direction, in contrast to the dispersed patterns of clean images. We hypothesize that this dominant shift, termed the Defense Direction, opposes the adversarial shift, pointing features back toward their correct class centers. Building on this insight, we propose Directional Bias-guided Defense (DBD), a test-time framework that estimates the Defense Direction and employs a DB-score-based two-stream reconstruction strategy to recover robust representations. Experiments on 15 datasets demonstrate that DBD not only achieves SOTA adversarial robustness while preserving clean accuracy, but also reveals the counterintuitive result that adversarial accuracy can even surpass clean accuracy. This demonstrates that adversarial perturbations inherently encode directional priors about the true decision boundary.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。