通过分析关键词影响,精准生成绕过内容安全检测的提示词。
TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysis
- 基于细粒度词影响分析,区分关键敏感词与检测差异词。
- 在3个主流模型上测试,平均绕过率提升54%以上。
- 适合研究模型安全漏洞或对抗攻击的开发者使用。
文本到图像(T2I)模型虽已能生成高质量图像,但可能产生不适宜工作场所(NSFW)内容,如色情、暴力、政治或歧视性内容。为降低风险,安全检查机制被引入以拦截潜在违规内容。然而,对抗性提示技术不断挑战这些检查机制的鲁棒性。本文提出TokenProber,一种面向敏感性感知的差分测试方法,用于评估T2I模型中拒绝机制的稳健性。核心思路是:对抗提示常因T2I模型与安全检查器对敏感内容的理解差异而成功。因此,我们对提示中特定词汇的影响进行细粒度分析,区分生成NSFW内容所必需的“脏词”与引发检测分歧的“差异词”。通过敏感性感知的变异策略,生成既能保留敏感内容生成能力又可规避检测的对抗提示。在3个主流T2I模型上,针对5个安全检查器,使用324个NSFW提示进行评估,结果显示,相比现有方法,其平均绕过率提升超过54%,充分揭示了现有拒绝机制的脆弱性。
原文摘要 · Abstract (English)
Text-to-image (T2I) models have significantly advanced in producing high-quality images. However, such models have the ability to generate images containing not-safe-for-work (NSFW) content, such as pornography, violence, political content, and discrimination. To mitigate the risk of generating NSFW content, refusal mechanisms, i.e., safety checkers, have been developed to check potential NSFW content. Adversarial prompting techniques have been developed to evaluate the robustness of the refusal mechanisms. The key challenge remains to subtly modify the prompt in a way that preserves its sensitive nature while bypassing the refusal mechanisms. In this paper, we introduce TokenProber, a method designed for sensitivity-aware differential testing, aimed at evaluating the robustness of the refusal mechanisms in T2I models by generating adversarial prompts. Our approach is based on the key observation that adversarial prompts often succeed by exploiting discrepancies in how T2I models and safety checkers interpret sensitive content. Thus, we conduct a fine-grained analysis of the impact of specific words within prompts, distinguishing between dirty words that are essential for NSFW content generation and discrepant words that highlight the different sensitivity assessments between T2I models and safety checkers. Through the sensitivity-aware mutation, TokenProber generates adversarial prompts, striking a balance between maintaining NSFW content generation and evading detection. Our evaluation of TokenProber against 5 safety checkers on 3 popular T2I models, using 324 NSFW prompts, demonstrates its superior effectiveness in bypassing safety filters compared to existing methods (e.g., 54%+ increase on average), highlighting TokenProber's ability to uncover robustness issues in the existing refusal mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。