arXiv:2512.00713cs.CRcs.AI2025-12被引 1

用语义概念做后门攻击,让视觉语言模型在识别特定概念时输出恶意内容。

Concept-Guided Backdoor Attack on Vision Language Models

  • 攻击基于语义概念而非像素,提升隐蔽性。
  • 两种方法均实现超90%攻击成功率,且对正常任务影响小。
  • 适合研究模型安全与对抗攻击的学者关注。

视觉语言模型(VLMs)在多模态文本生成中取得显著进展,但其快速应用引发安全漏洞担忧。现有针对VLM的后门攻击主要依赖显式像素级触发器或图像中的不可察觉扰动,虽有效却降低隐蔽性且易被图像防御检测。本文提出概念引导型后门攻击,从语义概念层面出发,引入两种新方法:第一种为概念阈值中毒(CTP),利用自然图像中的显式概念作为触发器,仅当目标概念出现时才触发恶意输出;第二种为概念瓶颈引导的未见后门攻击(CGUB),训练时借助概念瓶颈模型(CBM)干预内部概念激活,推理时丢弃CBM分支以保持原模型不变,从而系统性替换生成文本中的指定标签(如将"cat"替换为"dog"),即使该替换行为未在训练数据中出现。在多个VLM架构和数据集上的实验表明,两种方法均达到高攻击成功率,同时对正常任务性能影响较小。结果揭示了语义层面的漏洞是VLMs的新关键攻击面。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) have achieved impressive progress in multimodal text generation, yet their rapid adoption raises increasing concerns about security vulnerabilities. Existing backdoor attacks against VLMs primarily rely on explicit pixel-level triggers or imperceptible perturbations injected into images. While effective, these approaches reduce stealthiness and remain vulnerable to image-based defenses. We introduce concept-guided backdoor attacks, a new paradigm that operates at the semantic concept level rather than on raw pixels. We propose two different attacks. The first, Concept-Thresholding Poisoning (CTP), uses explicit concepts in natural images as triggers: only samples containing the target concept are poisoned, causing the model to behave normally in all other cases but consistently inject malicious outputs whenever the concept appears. The second, CBL-Guided Unseen Backdoor (CGUB), leverages a Concept Bottleneck Model (CBM) during training to intervene on internal concept activations, while discarding the CBM branch at inference time to keep the VLM unchanged. This design enables systematic replacement of a targeted label in generated text (for example, replacing "cat" with "dog"), even when the replacement behavior never appears in the training data. Experiments across multiple VLM architectures and datasets show that both CTP and CGUB achieve high attack success rates while maintaining moderate impact on clean-task performance. These findings highlight concept-level vulnerabilities as a critical new attack surface for VLMs.

后门攻击视觉语言模型语义安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。