通过净化提示词嵌入,防止文本生成有害图像。
Safe Text-to-Image Generation: Simply Sanitize the Prompt Embedding
- 在提示词嵌入中识别并打分有害词汇,动态过滤风险内容。
- 在5个基准测试中优于11种现有方法,生成图像质量高且鲁棒性强。
- 模块化设计可适配多种模型,适合需安全生成的开发者使用。
近年来,文本到图像(T2I)生成模型在生成与文本描述高度一致的高质量图像方面取得显著进展。然而,这些模型也存在生成不当内容的风险,可能产生违反使用政策的有害内容,如色情信息。现有安全生成方法通常通过从视觉表征中消除不希望的概念来抑制不当内容,却忽略了对文本表示的净化。尽管这些方法在一定程度上缓解了滥用风险,但在应对对抗性攻击时仍显脆弱。鉴于输入文本与输出图像之间的语义一致性是T2I模型的核心要求,我们发现文本表示很可能是导致不安全生成的主要来源。为此,我们提出嵌入净化器(ES),通过净化提示词嵌入中的不当概念来增强T2I模型的安全性。据我们所知,ES是首个可解释的安全生成框架,能够为提示词中的每个词元分配一个潜在危害得分。此外,ES采用即插即用的模块化设计,可无缝集成至多种T2I模型及其他防护机制。在五个提示词基准上的评估表明,ES优于十一种现有防护基线,实现了最先进的鲁棒性,同时保持了高质量的图像生成能力。
原文摘要 · Abstract (English)
In recent years, text-to-image (T2I) generation models have made significant progress in generating high-quality images that align with text descriptions. However, these models also face the risk of unsafe generation, potentially producing harmful content that violates usage policies, such as explicit material. Existing safe generation methods typically focus on suppressing inappropriate content by erasing undesired concepts from visual representations, while neglecting to sanitize the textual representation. Although these methods help mitigate the risk of misuse to some extent, their robustness remains insufficient when dealing with adversarial attacks. Given that semantic consistency between input text and output image is a core requirement of T2I models, we identify that textual representations are likely the primary source of unsafe generation. To this end, we propose Embedding Sanitizer (ES), which enhances the safety of T2I models by sanitizing inappropriate concepts in prompt embeddings. To our knowledge, ES is the first interpretable safe generation framework that assigns a score to each token in the prompt to indicate its potential harmfulness. In addition, ES adopts a plug-and-play modular design, offering compatibility for seamless integration with various T2I models and other safeguards. Evaluations on five prompt benchmarks show that ES outperforms eleven existing safeguard baselines, achieving state-of-the-art robustness while maintaining high-quality image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。