arXiv:2511.00446cs.CVcs.CR2025-11NeurIPS被引 4

用恶意文本污染CLIP预训练,实现高成功率后门攻击

ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training

  • 通过背景感知选择与驱动增强生成针对性恶意文本
  • 攻击成功率达95.83%,后门命中率98.68%
  • 针对文本模态的新型攻击,适合安全研究者参考

对比语言-图像预训练(CLIP)模型通过自监督对比学习,利用大规模网络数据对齐图像与文本对,显著推动了视觉-语言建模发展。然而,其依赖未经筛选的互联网数据,面临数据投毒与后门攻击风险。现有研究多聚焦图像攻击,而对同样关键的文本模态关注不足。本文提出ToxicTextCLIP框架,用于在预训练阶段生成高质量对抗性文本。该框架解决两大挑战:目标类别背景不一致导致的语义错位,以及背景一致文本稀缺。通过迭代应用:1)背景感知选择器,优先选取与目标类别背景一致的文本;2)背景驱动增强器,生成语义连贯且多样化的投毒样本。在分类与检索任务上的大量实验表明,ToxicTextCLIP实现最高95.83%的投毒成功率和98.68%的后门命中率(Hit@1),并成功绕过RoCLIP、CleanCLIP与SafeCLIP等防御机制。源代码可访问:https://github.com/xinyaocse/ToxicTextCLIP/

原文摘要 · Abstract (English)

The Contrastive Language-Image Pretraining (CLIP) model has significantly advanced vision-language modeling by aligning image-text pairs from large-scale web data through self-supervised contrastive learning. Yet, its reliance on uncurated Internet-sourced data exposes it to data poisoning and backdoor risks. While existing studies primarily investigate image-based attacks, the text modality, which is equally central to CLIP's training, remains underexplored. In this work, we introduce ToxicTextCLIP, a framework for generating high-quality adversarial texts that target CLIP during the pre-training phase. The framework addresses two key challenges: semantic misalignment caused by background inconsistency with the target class, and the scarcity of background-consistent texts. To this end, ToxicTextCLIP iteratively applies: 1) a background-aware selector that prioritizes texts with background content aligned to the target class, and 2) a background-driven augmenter that generates semantically coherent and diverse poisoned samples. Extensive experiments on classification and retrieval tasks show that ToxicTextCLIP achieves up to 95.83% poisoning success and 98.68% backdoor Hit@1, while bypassing RoCLIP, CleanCLIP and SafeCLIP defenses. The source code can be accessed via https://github.com/xinyaocse/ToxicTextCLIP/.

后门攻击CLIP文本投毒安全防护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。