arXiv:2608.10959cs.CVcs.CR2026-08

一句话让视觉语言模型任意生成指定文字,且无需重训练。

Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs

论文配图:Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs
图 1 · 摘自论文原文
  • 用多样化触发-文本对毒化模型,学习通用触发指令规则。
  • 可动态生成隐蔽触发,使模型输出任意未见过的描述文本。
  • 攻击隐蔽性强,兼容主流防御机制,适合研究安全漏洞者。

现有视觉语言模型(VLM)后门通常为静态漏洞:中毒阶段固定绑定特定触发与目标。我们提出一种可编程后门,单次中毒即可实现推理时任意选择未见过的目标文本,并按需生成隐蔽触发。该方法打破中毒与目标选择的绑定,无需重训练即可动态控制输出语义。核心包括:(1)启发式中毒策略,使模型学习通用‘触发即指令’规则;(2)基于特征空间的隐写技术,将任意目标文本映射为规范控制扰动或非语义补丁。插入任意图像后,被毒化的模型可生成与目标文本一致的输出,即使该目标从未在中毒阶段出现。大量实验表明,该攻击在任意到任意文本控制上成功率高,保持正常任务性能,并能绕过多种经典后门防御。

原文摘要 · Abstract (English)

Existing vision-language model (VLM) backdoors are usually treated as static vulnerabilities: one-to-one and N-to-N attacks bind one or more triggers to a finite set of targets before victim training. This assumption substantially underestimates the threat. We show that a single poisoning phase can implant a programmable backdoor into a VLM, allowing an attacker to choose previously unseen target-caption semantics at inference time and synthesize corresponding stealthy triggers on demand. Unlike fixed-mapping attacks, the proposed any-to-any caption-control paradigm decouples post-training target selection from poisoning, enabling dynamic control of target captions without retraining the VLM. Our method has two components. First, a heuristic poisoning strategy exposes the model to diverse trigger-caption pairs, encouraging it to learn a general trigger-as-instruction rule rather than memorize a specific backdoor pattern. Second, a feature-space trigger steganography method maps any attacker-specified target caption to a stealthy visual trigger, implemented as either a norm-controlled perturbation or a non-semantic patch. Once inserted into arbitrary images, these triggers cause the poisoned VLM to generate outputs semantically aligned with the chosen target caption, even when the target was unseen during poisoning. Extensive experiments show that our attack achieves high any-to-any caption-control success rates, preserves clean model utility, and remains effective under several classical backdoor defenses.

后门攻击视觉语言模型可编程控制隐写

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。