arXiv:2508.00591cs.CVcs.AI2025-08KDD被引 2

用中间生成步骤提前检测AI绘图中的不安全内容,又快又准。

Wukong Framework for Not Safe For Work Detection in Text-to-Image systems

  • 利用扩散模型早期去噪阶段的中间输出,实现快速检测。
  • 在三个数据集上准确率接近图像过滤器,但速度提升数倍。
  • 适合需要低延迟内容审核的AI绘图系统部署使用。

文本到图像(T2I)生成是主流AIGC技术,可生成多样创意图像,但部分输出可能包含暴力等不安全内容(NSFW),违反社区规范。现有外部防护分为两类:基于文本的过滤器分析用户提示,但忽略模型特异性且易受对抗攻击;基于图像的过滤器分析生成图像,计算成本高且引入延迟。扩散模型(如Stable Diffusion)通过带残差与注意力块的U-Net迭代去噪生成图像。我们观察到:(1)早期去噪步骤决定图像语义布局,(2)U-Net中的交叉注意力层对图文对齐至关重要。基于此,提出Wukong框架——一种基于Transformer的NSFW检测方法,利用早期去噪阶段中间输出,并复用预训练交叉注意力参数。该方法在扩散过程中实现早期检测,无需等待完整图像生成。同时构建新数据集,含提示、种子和图像级NSFW标签,评估结果表明:Wukong显著优于文本过滤器,在准确率上媲美图像过滤器,效率大幅提升。

原文摘要 · Abstract (English)

Text-to-Image (T2I) generation is a popular AI-generated content (AIGC) technology enabling diverse and creative image synthesis. However, some outputs may contain Not Safe For Work (NSFW) content (e.g., violence), violating community guidelines. Detecting NSFW content efficiently and accurately, known as external safeguarding, is essential. Existing external safeguards fall into two types: text filters, which analyze user prompts but overlook T2I model-specific variations and are prone to adversarial attacks; and image filters, which analyze final generated images but are computationally costly and introduce latency. Diffusion models, the foundation of modern T2I systems like Stable Diffusion, generate images through iterative denoising using a U-Net architecture with ResNet and Transformer blocks. We observe that: (1) early denoising steps define the semantic layout of the image, and (2) cross-attention layers in U-Net are crucial for aligning text and image regions. Based on these insights, we propose Wukong, a transformer-based NSFW detection framework that leverages intermediate outputs from early denoising steps and reuses U-Net's pre-trained cross-attention parameters. Wukong operates within the diffusion process, enabling early detection without waiting for full image generation. We also introduce a new dataset containing prompts, seeds, and image-specific NSFW labels, and evaluate Wukong on this and two public benchmarks. Results show that Wukong significantly outperforms text-based safeguards and achieves comparable accuracy of image filters, while offering much greater efficiency.

内容安全扩散模型图像生成实时检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。