arXiv:2511.01317cs.CV2025-11

用CLIP模型生成几乎看不见的攻击图像,骗过分类模型

A Generative Adversarial Approach to Adversarial Attacks Guided by Contrastive Language-Image Pre-trained Model

  • 结合CLIP语义对齐能力,用文本引导生成对抗扰动
  • 在多目标场景下欺骗多标签分类器,保持原图结构相似性
  • 适合研究模型安全或对抗攻击的读者

深度学习模型虽强大,但易受对抗攻击影响。本文提出一种基于生成对抗网络的攻击方法,利用CLIP模型将文本语义融入损失函数,生成视觉上难以察觉却极具破坏力的对抗扰动。该方法结合了基于显著性的自编码器(SSAE)的集中扰动策略与生成对抗多场景攻击(GAMA)中相似文本嵌入的思想,能在复杂多目标环境中精准操纵场景,有效欺骗多标签分类模型。实验在多种黑盒模型上验证,结果表明该方法性能优于或相当现有技术,同时保持更高的图像视觉保真度。

原文摘要 · Abstract (English)

The rapid growth of deep learning has brought about powerful models that can handle various tasks, like identifying images and understanding language. However, adversarial attacks, an unnoticed alteration, can deceive models, leading to inaccurate predictions. In this paper, a generative adversarial attack method is proposed that uses the CLIP model to create highly effective and visually imperceptible adversarial perturbations. The CLIP model's ability to align text and image representation helps incorporate natural language semantics with a guided loss to generate effective adversarial examples that look identical to the original inputs. This integration allows extensive scene manipulation, creating perturbations in multi-object environments specifically designed to deceive multilabel classifiers. Our approach integrates the concentrated perturbation strategy from Saliency-based Auto-Encoder (SSAE) with the dissimilar text embeddings similar to Generative Adversarial Multi-Object Scene Attacks (GAMA), resulting in perturbations that both deceive classification models and maintain high structural similarity to the original images. The model was tested on various tasks across diverse black-box victim models. The experimental results show that our method performs competitively, achieving comparable or superior results to existing techniques, while preserving greater visual fidelity.

对抗攻击CLIP生成模型图像安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。