arXiv:2508.11904cs.CV2025-08

通过检测-抑制机制实现图文生成的安全控制,兼顾安全与图像质量。

SafeCtrl: Region-Based Safety Control for Text-to-Image Diffusion via Detect-Then-Suppress

  • 先定位有害区域,再抑制语义而非替换内容
  • 在多个数据集上安全率超90%,保真度损失低于15%
  • 无需像素级标注,适合快速部署到现有模型

文本到图像模型的广泛应用面临生成有害内容的风险。现有安全方法如提示重写或模型微调常在安全性和保真度间权衡。基于定位的方法虽有潜力,但依赖显式的“概念替换”可能导致语义不一致。为此,我们提出SafeCtrl,一种轻量、非侵入式插件,先精确定位不安全内容,再通过抑制有害语义而非硬性替换,使生成过程自然演化为安全且上下文相关的替代结果。关键创新在于使用直接偏好优化(DPO)训练策略,利用现成的图像级偏好数据,无需昂贵的像素级标注即可学习精细的抑制行为,在推理时实现区域引导干预。大量实验表明,SafeCtrl在安全有效性与保真度保持方面显著优于当前最优方法。研究提示,解耦的抑制式控制是构建更负责任生成模型的有效且可扩展方向。

原文摘要 · Abstract (English)

The widespread deployment of text-to-image models is challenged by their potential to generate harmful content. While existing safety methods, such as prompt rewriting or model fine-tuning, provide valuable interventions, they often introduce a trade-off between safety and fidelity. Recent localization-based approaches have shown promise, yet their reliance on explicit ``concept replacement" can sometimes lead to semantic incongruity. To address these limitations, we explore a more flexible detect-then-suppress paradigm. We introduce SafeCtrl, a lightweight, non-intrusive plugin that first precisely localizes unsafe content. Instead of performing a hard A-to-B substitution, SafeCtrl then suppresses the harmful semantics, allowing the generative process to naturally and coherently resolve into a safe, context-aware alternative. A key aspect of our work is a novel training strategy using Direct Preference Optimization (DPO). We leverage readily available, image-level preference data to train our module, enabling it to learn nuanced suppression behaviors and perform region-guided interventions at inference without requiring costly, pixel-level annotations. Extensive experiments show that SafeCtrl significantly outperforms state-of-the-art methods in both safety efficacy and fidelity preservation. Our findings suggest that decoupled, suppression-based control is a highly effective and scalable direction for building more responsible generative models.

安全控制扩散模型图像生成抑制机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。