用少量私有令牌提升ViT模型抗攻击能力，不伤原有性能
Robustness Tokens: Towards Adversarial Robustness of Transformers
- 在Transformer中添加少量私有令牌进行微调，计算开销低
- 显著增强视觉Transformer对白盒攻击的鲁棒性
- 适合需保护模型安全的下游应用部署
大型预训练基础模型被广泛用于各类任务。由于这些模型公开可用，直接将其作为下游任务的主干网络可能面临由相同公开模型生成的对抗攻击。本文提出一种针对Transformer架构的新型方法——鲁棒性令牌（Robustness Tokens），通过微调少量额外的私有令牌实现抗干扰能力提升，而无需像传统对抗训练那样调整模型参数。实验表明，该方法使视觉Transformer模型在保持原有下游性能的同时，显著增强了对白盒对抗攻击的鲁棒性。
原文摘要 · Abstract (English)
Recently, large pre-trained foundation models have become widely adopted by machine learning practitioners for a multitude of tasks. Given that such models are publicly available, relying on their use as backbone models for downstream tasks might result in high vulnerability to adversarial attacks crafted with the same public model. In this work, we propose Robustness Tokens, a novel approach specific to the transformer architecture that fine-tunes a few additional private tokens with low computational requirements instead of tuning model parameters as done in traditional adversarial training. We show that Robustness Tokens make Vision Transformer models significantly more robust to white-box adversarial attacks while also retaining the original downstream performances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。