通过改写提示词，让图像生成更安全且不被恶意触发。
PEPPER: Perception-Guided Perturbation for Robust Backdoor Defense in Text-to-Image Diffusion Models
- 用语义不同但视觉相似的提示词替换原输入，干扰攻击触发。
- 在多个攻击下成功率降低超过90%，生成质量基本不变。
- 无需训练或模型权重，可与现有防御方法配合使用。
近期研究发现，文本到图像（T2I）扩散模型易受后门攻击，输入提示中的特定触发词可引导生成有害或非预期内容。除了触发词本身，攻击效果还会扩散至文本嵌入空间中的邻近词汇。为此，我们提出PEPPER（感知引导扰动），一种无需训练或访问模型权重的防御机制。该方法将原始提示重写为语义上差异大但视觉上相似的新提示，并加入不易察觉的元素。此策略可破坏输入提示中嵌入的触发信号，脱离被攻击的邻域，从而显著提升鲁棒性。实验表明，PEPPER在针对文本编码器的攻击下表现尤为出色，大幅降低攻击成功率，同时保持生成质量。此外，它可与任意现有防御方法结合，实现更强且更具泛化能力的防护效果。
原文摘要 · Abstract (English)
Recent studies show that text-to-image (T2I) diffusion models are vulnerable to backdoor attacks, where a trigger in the input prompt can steer generation toward harmful or unintended content. Beyond the trigger token itself, backdoor effects can spread to neighboring tokens in the text embedding space. To address this, we introduce PEPPER (PErcePtion-Guided PERturbation), a backdoor defense that rewrites the caption into a semantically distant yet visually similar caption while adding unobtrusive elements. With this strategy, PEPPER disrupts the trigger embedded in the input prompt, escapes the attacked neighborhood, and thereby achieves enhanced robustness without training or access to model weights. Experiments show that PEPPER is particularly effective against text encoder-based attacks, substantially reducing attack success while preserving generation quality. PEPPER can also be paired with any existing defenses yielding consistently stronger and generalizable robustness than any standalone method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。