arXiv:2501.03544cs.CVcs.AI2025-01中稿 · publication in IEE…被引 17

用可学习的软提示实现文本生成图像的安全过滤,快且准。

PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models

  • 设计一个隐式系统提示,在文本嵌入空间优化安全软提示。
  • 在5个数据集上将不安全内容生成率压至6%以下,速度比之前快3.8倍。
  • 无需额外模型,适合部署在真实场景中保护AI图像生成安全。

近期文本到图像(T2I)模型在生成高质量图像方面表现卓越,但易被滥用生成不适宜工作场合(NSFW)内容,如色情、暴力、政治敏感及令人不适的图像,引发严重伦理问题。本文提出PromptGuard,一种受大语言模型系统提示机制启发的内容安全调控技术。不同于LLMs,T2I模型缺乏直接的行为规范接口。核心思想是优化一个安全软提示(P*),作为隐式系统提示嵌入于T2I模型的文本嵌入空间中,以通用方式调节不安全输入,实现安全且真实的图像生成,不影响推理效率,也无需代理模型。通过分治策略优化类别特定软提示并融合为统一安全引导,提升可靠性与实用性。跨五个数据集的大量实验表明,PromptGuard有效抑制了不安全内容生成,同时保持高质量良性输出。相比先前方法,其速度提升3.8倍,优于八种主流防御方案。多头安全分类器与基于视觉语言模型的护栏评估均验证其鲁棒性,平均不安全比例分别为5.84%和6.18%。代码与数据集已公开于https://t2i-promptguard.github.io/。

原文摘要 · Abstract (English)

Recent text-to-image (T2I) models have exhibited remarkable performance in generating high-quality images from text descriptions. However, these models are vulnerable to misuse, particularly generating not-safe-for-work (NSFW) content, such as sexually explicit, violent, political, and disturbing images, raising serious ethical concerns. In this work, we present PromptGuard, a novel content moderation technique that draws inspiration from the system prompt mechanism in large language models (LLMs) for safety alignment. Unlike LLMs, T2I models lack a direct interface for enforcing behavioral guidelines. Our key idea is to optimize a safety soft prompt that functions as an implicit system prompt within the T2I model's textual embedding space. This universal soft prompt (P*) directly moderates NSFW inputs, enabling safe yet realistic image generation without affecting inference efficiency or requiring proxy models. We further enhance its reliability and helpfulness through a divide-and-conquer strategy that optimizes category-specific soft prompts and combines them into unified safety guidance. Extensive experiments across five datasets demonstrate that PromptGuard effectively mitigates NSFW content generation while preserving high-quality benign outputs. PromptGuard is 3.8 times faster than prior content moderation methods while outperforming eight state-of-the-art defenses. Evaluations using both a multi-head safety classifier and a VLM-based guardrail further confirm its robustness, with average unsafe ratios of 5.84% and 6.18%, respectively. Our code and dataset are available at https://t2i-promptguard.github.io/.

图像生成内容安全软提示AIGC

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。