提出无痕迹后门攻击,让文本生成图像模型在隐蔽中被操控
Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Models
- 用语法结构作触发器,让提示词变化时输出更敏感
- 97.5%攻击成功率,98%以上绕过主流检测方法
- 适合研究模型安全与对抗攻击的学者参考
针对文本到图像扩散模型的后门攻击已迅速发展。然而,现有后门样本常表现出两个显著异常:1)语义一致性,即不同提示词生成的图像语义相似;2)注意力一致性,即触发器在交叉注意力图中引发一致的结构响应。这些一致性特征易被防御方检测。本文提出无痕迹后门攻击(TwT),通过显式削弱上述一致性实现隐蔽攻击。具体而言,利用语法结构作为触发器,增强对提示词变化的敏感性,打破语义一致性;同时引入基于核最大均值差异(KMMD)的正则化方法,对齐后门与正常样本的交叉注意力分布,破坏注意力一致性。大量实验表明,该方法达到97.5%攻击成功率,平均超过98%的后门样本可绕过三种先进检测机制,暴露当前防御方案的脆弱性。代码已公开于https://github.com/Robin-WZQ/TwT。
原文摘要 · Abstract (English)
Backdoor attacks targeting text-to-image diffusion models have advanced rapidly. However, current backdoor samples often exhibit two key abnormalities compared to benign samples: 1) Semantic Consistency, where backdoor prompts tend to generate images with similar semantic content even with significant textual variations to the prompts; 2) Attention Consistency, where the trigger induces consistent structural responses in the cross-attention maps. These consistencies leave detectable traces for defenders, making backdoors easier to identify. In this paper, toward stealthy backdoor samples, we propose Trigger without Trace (TwT) by explicitly mitigating these consistencies. Specifically, our approach leverages syntactic structures as backdoor triggers to amplify the sensitivity to textual variations, effectively breaking down the semantic consistency. Besides, a regularization method based on Kernel Maximum Mean Discrepancy (KMMD) is proposed to align the distribution of cross-attention responses between backdoor and benign samples, thereby disrupting attention consistency. Extensive experiments demonstrate that our method achieves a 97.5% attack success rate while exhibiting stronger resistance to defenses. It achieves an average of over 98% backdoor samples bypassing three state-of-the-art detection mechanisms, revealing the vulnerabilities of current backdoor defense methods. The code is available at https://github.com/Robin-WZQ/TwT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。