arXiv:2609.04721cs.LG2026-09

发现语言模型拒绝行为在不同架构中共享同一方向,可跨模型迁移。

Locating and Steering Refusal Beyond Attention

  • 用刚性旋转对齐不同模型的表示空间,实现拒绝方向跨架构传递。
  • 在四种架构上验证,该方向能有效降低越狱攻击成功率。
  • 只需在各架构的输出写入点重新估计方向,无需重构建模。

语言模型中的拒绝行为存在于变换器的残差流中一个特定方向,这一发现已广泛用于安全与可解释性工具。状态空间模型(SSM)通过递归更新传递信息,不依赖注意力机制,其与变换器无共享的令牌混合结构。然而,该安全表征仍能跨架构存活:仅需一次刚性旋转即可对齐两模型的表示空间,使二者真正共享该方向。在变换器上训练的有害性探测器可识别SSM中的有害输入;移除该对齐方向后,模型不再拒绝原本应拒绝的攻击,而随机方向影响甚微。关键在于,危害信号可在每一层的输出写入点被清晰读取,而非加法后的残差流中。控制实验表明,干预位置比作用位置更重要。通过检测器触发门控机制,该方向在四类架构(SSM、变换器、循环、混合)中均显著降低越狱成功率,且在对抗性提示调优下仍有效。门控规则简单固定,证明跨架构转移的是方向本身,而非防御强度。因此,基于拒绝的安全工具仅需在新架构的写入点重新估计方向,即可复用。

原文摘要 · Abstract (English)

Where inside a language model does refusal live, and does that place change when the architecture does? In a transformer, refusal is governed by a single direction in the residual stream, a finding that safety and interpretability tooling now depend on. State-space models (SSMs) route information through a recurrent update instead of attention, sharing no token-mixing mechanism with a transformer. Does the same safety representation survive this shift, or must it be rediscovered per architecture? It survives. A single rigid rotation, which can only reorient a space and not reshape it, aligns one model's representation space with another's, so the two genuinely share the representation. A harm probe trained on a transformer then flags an SSM's harmful inputs, and removing the aligned direction makes a model answer attacks it would otherwise refuse, while a random direction of the same size does far less. What is architecture-specific is not where the direction is steered but where it must be read. Each layer computes a fresh output that is then added into the residual stream, and harm is cleanly readable at this output, the write site, before the addition. A control that holds the intervention's strength fixed shows that what matters is where the direction is estimated, not where it is applied. Applied through a detector-triggered gate, this direction lowers jailbreak success in all four architecture families we test (SSM, transformer, recurrent, hybrid), and on the SSM it holds against an attacker that tunes its prompt against the defense. The gate only matches a trivial rule that returns a fixed refusal whenever the same detector fires, so what transfers across architectures is the direction itself, not defense strength. Safety tooling built on refusal therefore ports to a new architecture by re-estimating the direction at that architecture's write site, not by rebuilding it.

安全对齐模型可解释性跨架构迁移拒绝机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。