arXiv:2606.06875cs.CVcs.CR2026-06

提出UVR框架,让多模态扩散模型生成更安全的图像。

Unified Safe In-context Image Generation in Multimodal Diffusion Transformers via Restricting Unsafe Information Flows

论文配图:Unified Safe In-context Image Generation in Multimodal Diffusion Transformers via Restricting Unsafe Information Flows
图 1 · 摘自论文原文
  • 通过分析注意力流动,定位生成图像中不安全内容出现的阶段。
  • 在91%和77%的生成与编辑任务中有效消除有害内容。
  • 无需训练即可应用,适合追求图像安全性的开发人员使用。

具备多模态注意力(MM-Attn)的扩散变换器(DiTs)已成为图像生成的主流范式。然而,在图像到图像(I2I)编辑任务中,防止生成有害内容仍是关键挑战。现有安全机制主要针对文本到图像(T2I)合成或U-Net架构,难以有效应用于基于DiT的统一安全防护。为此,我们提出无需训练的统一视觉安全调节器(UVR),通过调控生成图像中的不安全语义来实现安全生成。UVR基于对MM-Attn中信息流的注意力动态分析,发现一个与任务无关的启动阶段:此时不安全语义在输出图像块中迅速出现并可精准定位;随后进入任务相关的语义放大与干扰阶段,有害信号被进一步传播并混入正常内容。基于此,UVR通过统一、针对性的注意力调制及对不安全输出块中危害性信息流的显式限制,实现安全防护。在多种概念上的实验表明,UVR在图像生成与编辑任务中分别达到91%和77%的清除率,同时保持良好的视觉质量和保真度,性能损失极小。代码已开源:https://github.com/deng12yx/UVR。

原文摘要 · Abstract (English)

Diffusion transformers (DiTs) equipped with multimodal attention (MM-Attn) have become a dominant paradigm for image generation. However, preventing the generation of harmful content remains a critical challenge, particularly in image-to-image (I2I) editing tasks. Existing safety mechanisms are primarily designed for text-to-image (T2I) synthesis or U-Net-based architectures, which limits their effectiveness for unified safety mitigation in DiT-based frameworks. To bridge this gap, we propose Unified Visual Safety Regulator (UVR), a training-free safe generation framework that regulates unsafe semantics in generated images. UVR is grounded in an analysis of attention dynamics from the perspective of information flow in MM-Attn. We identify a task-independent start-up stage, during which unsafe semantics in output patches rapidly emerge and can be accurately localized, followed by task-specific semantic amplification and interference stages, where harmful signals are further propagated and entangled with benign content. Based on these observations, UVR mitigates unsafe generation through unified, targeted attention modulation and explicit restriction of harmful information flow over the identified unsafe output patches. Experiments across various concepts show that UVR achieves state-of-the-art safety performance by achieving 91% and 77% erase rate in image synthesis and editing tasks, while preserving visual quality and fidelity with minimal degradation. Code is available at https://github.com/deng12yx/UVR.

图像生成扩散模型安全机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。