不训练不更新,通过调整模型内部表示实现视频生成安全对齐。
Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steering

- 在推理时通过调节中间层隐藏状态方向,引导生成安全内容。
- 仅用一个方向即可区分安全与危险生成路径,效果显著且开销极小。
- 适用于多种模型和生成任务,适合关注生成安全性的研究者使用。
开放权重的视频扩散模型可生成暴力、虚假信息等不安全内容,现有防御方法要么需昂贵的安全微调导致通用能力下降,要么依赖外部过滤器,易被对抗性提示绕过。本文提出REINS(Representation-space Inference-time Safety steering),一种无需训练的方法,在推理时通过引导模型内部表示向安全方向调整。关键发现是:安全相关结构在线性编码于视频扩散变压器的隐藏状态激活中,通过监督主成分分析(Supervised PCA)在二元安全标签上发现单一方向,即可有效分离安全与不安全生成轨迹。推理时,将该方向添加至中间变压器层的隐藏状态,即可将生成内容从有害内容转向语义相关的安全替代内容,无需权重更新、无需概念枚举,计算开销可忽略。机制分析表明,虽然安全信息随变压器深度单调积累,但扰动效果在中间层(约50%深度)达到峰值,揭示了信息可用性与下游传播能力之间的根本权衡。我们在9个视频扩散模型、多个参数规模(1.3B-5B)及文本到视频、图像到视频生成任务上评估REINS,据我们所知,这是视频生成领域最广泛的安全部署评估。
原文摘要 · Abstract (English)
Open-weight video diffusion models can generate photorealistic unsafe content, from violence to misinformation, yet existing defenses either require expensive safety fine-tuning that degrades general capability, or apply external filters that are trivially bypassed by adversarial prompts. We present REINS (REpresentation-space INference-time Safety steering), a training-free method that aligns video diffusion models at inference time by steering their internal representations toward safe generation. Our key finding is that safety-relevant structure is linearly encoded in the hidden-state activations of video diffusion transformers, and a single direction, discovered via Supervised PCA on binary safety labels, suffices to separate safe from unsafe generation trajectories. At inference, adding this direction to hidden states at an intermediate transformer layer redirects generation from harmful content to semantically related safe alternatives, with no weight updates, no concept enumeration, and negligible computational overhead. Through mechanistic analysis, we reveal that while safety information accumulates monotonically with transformer depth, steering effectiveness peaks at intermediate layers (~50% depth), exposing a fundamental tradeoff between information availability and downstream propagation capacity. We evaluate REINS across 9 video diffusion models, multiple parameter scales (1.3B-5B), and both text-to-video and image-to-video generation, to our knowledge, the broadest safety evaluation suite in the video generation literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。