无需训练即可实时过滤文本生成中的有害内容,保持图像质量。
SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation
- 通过检测文本嵌入空间的有害概念子空间并引导远离,实现无训练过滤。
- 在不降低图像质量前提下,有效抑制90%以上有害内容生成。
- 适合需快速部署安全机制的AI生成平台或应用开发人员。
扩散模型在生成高质量图像和视频方面取得显著进展,但也增加了生成有害内容的风险。现有基于遗忘/编辑的方法虽可移除有害概念,但存在三大问题:(1) 无法在不训练的情况下即时移除有害概念;(2) 安全生成能力依赖于训练数据;(3) 修改模型权重可能影响非有害内容的质量。为此,我们提出SAFREE,一种无需训练且可自适应的T2I与T2V安全生成新方法,不改变模型权重。具体而言,我们在文本嵌入空间中检测一组有害概念对应的子空间,并将提示嵌入引导远离该子空间,从而过滤有害内容同时保留原始语义。为平衡毒性过滤与安全概念保留之间的权衡,SAFREE引入新颖的自验证过滤机制,在应用过滤后的嵌入时动态调整去噪步数。此外,我们还在扩散潜在空间中引入自适应重注意力机制,选择性削弱与有害概念相关的像素级特征。最终,SAFREE实现了连贯的安全检测,保障输出的一致性、保真度与安全性。在无训练基线中,SAFREE在抑制不安全内容方面达到最先进性能,能有效过滤目标概念并保持高质量图像;其表现也与训练型方法相当。我们将SAFREE扩展至多种T2I骨干网络与T2V任务,证明其灵活性与泛化能力。SAFREE为确保视觉生成安全提供了强大且可适应的防护方案。
原文摘要 · Abstract (English)
Recent advances in diffusion models have significantly enhanced their ability to generate high-quality images and videos, but they have also increased the risk of producing unsafe content. Existing unlearning/editing-based methods for safe generation remove harmful concepts from models but face several challenges: (1) They cannot instantly remove harmful concepts without training. (2) Their safe generation capabilities depend on collected training data. (3) They alter model weights, risking degradation in quality for content unrelated to toxic concepts. To address these, we propose SAFREE, a novel, training-free approach for safe T2I and T2V, that does not alter the model's weights. Specifically, we detect a subspace corresponding to a set of toxic concepts in the text embedding space and steer prompt embeddings away from this subspace, thereby filtering out harmful content while preserving intended semantics. To balance the trade-off between filtering toxicity and preserving safe concepts, SAFREE incorporates a novel self-validating filtering mechanism that dynamically adjusts the denoising steps when applying the filtered embeddings. Additionally, we incorporate adaptive re-attention mechanisms within the diffusion latent space to selectively diminish the influence of features related to toxic concepts at the pixel level. In the end, SAFREE ensures coherent safety checking, preserving the fidelity, quality, and safety of the output. SAFREE achieves SOTA performance in suppressing unsafe content in T2I generation compared to training-free baselines and effectively filters targeted concepts while maintaining high-quality images. It also shows competitive results against training-based methods. We extend SAFREE to various T2I backbones and T2V tasks, showcasing its flexibility and generalization. SAFREE provides a robust and adaptable safeguard for ensuring safe visual generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。