通过动态调节防御强度,让AI绘图更安全且不伤好图质量。
PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generation
- 用大模型重写不安全提示为安全等价版本,训练轻量软提示
- 推理时根据提示危险度自适应开启防御,降低误伤好图风险
- 在多个模型上实现2.36%的最低不安全生成率,抗攻击能力强
文本到图像(T2I)模型虽具强大生成能力,却易产出不当内容。现有防护多依赖大规模图文数据集,静态添加防御标记,导致计算开销大、正常图像质量下降,且难以适配真实提示的复杂安全需求。为此,我们提出PromptSafe,一种轻量级门控提示调优框架:先用大模型将不安全提示转换为语义一致的安全版本,构建纯文本训练数据;再优化一个通用软提示,在扩散去噪过程中排斥不安全嵌入、吸引安全嵌入。为避免过度抑制正常提示,引入门控机制,依据估计的提示毒性动态调整防御强度,实现风险匹配保护。实验表明,PromptSafe在多个基准和模型上达到2.36%的最低不安全生成率,同时保持高正常生成保真度,并具备对未见有害类别、跨模型架构的强泛化与抗自适应攻击能力,适用于实际安全部署。
原文摘要 · Abstract (English)
Text-to-image (T2I) models have demonstrated remarkable generative capabilities but remain vulnerable to producing not-safe-for-work (NSFW) content, such as violent or explicit imagery. While recent moderation efforts have introduced soft prompt-guided tuning by appending defensive tokens to the input, these approaches often rely on large-scale curated image-text datasets and apply static, one-size-fits-all defenses at inference time. However, this results not only in high computational cost and degraded benign image quality, but also in limited adaptability to the diverse and nuanced safety requirements of real-world prompts. To address these challenges, we propose PromptSafe, a gated prompt tuning framework that combines a lightweight, text-only supervised soft embedding with an inference-time gated control network. Instead of training on expensive image-text datasets, we first rewrite unsafe prompts into semantically aligned but safe alternatives using an LLM, constructing an efficient text-only training corpus. Based on this, we optimize a universal soft prompt that repels unsafe and attracts safe embeddings during the diffusion denoising process. To avoid over-suppressing benign prompts, we introduce a gated mechanism that adaptively adjusts the defensive strength based on estimated prompt toxicity, thereby aligning defense intensity with prompt risk and ensuring strong protection for harmful inputs while preserving benign generation quality. Extensive experiments across multiple benchmarks and T2I models show that PromptSafe achieves a SOTA unsafe generation rate (2.36%), while preserving high benign fidelity. Furthermore, PromptSafe demonstrates strong generalization to unseen harmful categories, robust transferability across diffusion model architectures, and resilience under adaptive adversarial attacks, highlighting its practical value for safe and scalable deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。