让文生图模型更安全:通过稀疏特征迁移实现跨域鲁棒性防护。
Robust and Generalizable Safety Steering for Text-to-Image Diffusion Transformers

- 基于位置感知的稀疏编码器,定位稳定干预点。
- 在FLUX.1和SD3.5上降低90%以上目标域不安全生成率。
- 适合需要跨场景安全控制的AI图像生成系统开发者。
扩散变压器已成为文生图生成的强大骨干网络,但其分层与跨模态生成过程使安全控制本质不同于提示层过滤或输出层检测。有害语义可能在文本表示中表达微弱,逐步嵌入视觉隐变量,并最终与渲染动态纠缠。因此,固定层的安全引导易不稳定,且从已知风险学习的机制难以可靠迁移到新风险域。本文提出SafeDIG,将DiT安全适配建模为位置感知的稀疏特征迁移。首先,在功能各异的DiT干预位置构建稀疏自编码器,并通过鲁棒性感知预训练路由,优先选择在源-目标风险转移下仍稳定的干预点。随后,冻结SAE编码器作为可复用的稀疏安全字典,仅适应解码器以匹配目标域激活流形,分离可迁移安全特征与域特定激活几何。推理时,SafeDIG结合混合(Blend)与排斥(Repel)操作,将不安全激活导向迁移的安全流形或远离有害稀疏方向。在FLUX.1 Dev与Stable Diffusion 3.5 Large上的实验表明,SafeDIG持续降低目标域及总体不安全生成率,同时保持源域安全性和图像质量。
原文摘要 · Abstract (English)
Diffusion Transformers have become a powerful backbone for text-to-image generation, but their layered and cross-modal generation process makes safety control fundamentally different from prompt-level filtering or output-level detection. Harmful semantics may be weakly expressed in text representations, progressively bound to visual latents, and finally entangled with rendering dynamics. As a result, safety steering at a fixed layer can be unstable, and a steering mechanism learned from known risks may not transfer reliably to a shifted target risk domain. We propose SafeDIG, a safety steering framework that formulates DiT safety adaptation as position-aware sparse feature transfer. SafeDIG first constructs Sparse Autoencoders over functionally distinct DiT intervention positions and uses robustness-aware pre-training routing to prioritize intervention sites that are expected to remain stable under source-target risk shift. It then separates transferable safety features from domain-specific activation geometry by freezing the SAE encoder as a reusable sparse safety dictionary and adapting only the decoder to the target-domain activation manifold. During inference, SafeDIG combines Blend and Repel operations to steer unsafe activations toward transferred safety manifolds or away from harmful sparse directions. Experiments on FLUX.1 Dev and Stable Diffusion 3.5 Large show that SafeDIG consistently reduces target-domain and overall unsafe generation rates while preserving source-domain safety and image quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。