提出高效通用对抗攻击方法，速度提升23倍且效果更好
Efficient and Effective Universal Adversarial Attack against Vision-Language Pre-training Models
- 直接优化生成通用扰动，避免耗时的模型训练
- 在三大数据集上攻击成功率超现有方法，耗时减少23倍
- 适合快速评估视觉语言模型安全性的研究人员
视觉语言预训练（VLP）模型在多类下游任务中广泛应用，但其易受对抗攻击威胁。传统非通用攻击虽有效却计算开销大，不适用于实时场景；现有基于生成器的通用对抗扰动（UAP）方法也耗时严重。为此，本文提出直接优化型通用攻击方法DO-UAP，显著降低资源消耗同时保持高攻击性能。我们研究了多模态损失设计的必要性，并引入有效的数据增强策略。在三个基准VLP数据集、六种主流VLP模型及三类经典下游任务上的大量实验表明，DO-UAP将时间消耗降低23倍，攻击成功率更高。
原文摘要 · Abstract (English)
Vision-language pre-training (VLP) models, trained on large-scale image-text pairs, have become widely used across a variety of downstream vision-and-language (V+L) tasks. This widespread adoption raises concerns about their vulnerability to adversarial attacks. Non-universal adversarial attacks, while effective, are often impractical for real-time online applications due to their high computational demands per data instance. Recently, universal adversarial perturbations (UAPs) have been introduced as a solution, but existing generator-based UAP methods are significantly time-consuming. To overcome the limitation, we propose a direct optimization-based UAP approach, termed DO-UAP, which significantly reduces resource consumption while maintaining high attack performance. Specifically, we explore the necessity of multimodal loss design and introduce a useful data augmentation strategy. Extensive experiments conducted on three benchmark VLP datasets, six popular VLP models, and three classical downstream tasks demonstrate the efficiency and effectiveness of DO-UAP. Specifically, our approach drastically decreases the time consumption by 23-fold while achieving a better attack performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。