arXiv:2606.05290cs.CVcs.AI2026-06

安全表征可跨模型迁移,用单一方向控制多类生成模型的安全性。

Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation

论文配图:Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation
图 1 · 摘自论文原文
  • 从源模型提取安全潜空间方向,轻量对齐后移植到目标模型。
  • 无需目标端危险数据,仍实现与本地训练相当的安全提升效果。
  • 支持细粒度控制,适合希望快速部署通用安全机制的研究者。

生成模型的安全控制仍是核心挑战,现有方法多依赖特定模型,需重新训练或定制干预。本文首次探索安全是否可作为可移植的潜空间方向:在源大语言模型中通过成对的安全-不安全提示学习安全方向,仅用良性数据进行轻量对齐,将方向传输至目标生成器并在推理时应用。关键在于,目标端全程不接触危险数据,以检验安全能否通过共享表示几何传递。除全局方向外,还提出多向量扩展,捕捉类别特定的安全行为,实现更精细控制。在文本到图像、文本到视频生成中测试多种异构模型组合,结果表明:迁移后的安全方向在降低攻击成功率(ASR)的同时,保持与使用危险数据本地训练相当的CLIP分数和FID表现,且无需目标侧危险数据。这表明安全改进不以生成质量为代价。研究揭示安全行为具有模块化特征——并非仅限于模型内部,而是可通过跨模型共有的潜空间方向进行调控,为无需目标端危险数据的轻量级、可复用安全机制提供了新路径。

原文摘要 · Abstract (English)

Recent progress in generative modeling has made safety control a central challenge, yet existing approaches remain largely model-specific, requiring retraining or tailored interventions for each new architecture. In this work, we ask whether safety can be represented as a portable latent direction, learned once and reused across heterogeneous generators. We introduce the first framework for cross-model safety steering, in which a safety direction is estimated in a source LLM from paired safe-unsafe prompts, transported to a target generator through a lightweight alignment fitted on benign data alone, and applied at inference time. Crucially, our pipeline never accesses unsafe data on the target side, isolating whether safety can be transferred through shared representation geometry. Beyond a single global direction, we also identify a multi-vector extension that captures category-specific safety behaviors, enabling more selective control. We evaluate our approach in text-to-image and text-to-video generation across diverse source-target model pairs. Across models, transferred safety directions achieve ASR reduction and CLIP-Score/FID trade-offs comparable to directions learned natively on the target model using unsafe data, while requiring no target-side unsafe data. This indicates that safety improvements do not come at the expense of generation quality. Our results point to a modular view of safety: safety-relevant behavior is not purely model-local, but can be controlled through latent directions that persist across models. This suggests a new path toward lightweight, reusable safety mechanisms that do not require target-side unsafe data.

安全生成跨模型潜空间可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。