比较两种模型拒绝机制干预方法,发现反事实翻转效果接近经典方法。
Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP

- 用差异均值和INLP两种方法干预模型拒绝行为
- 反事实翻转在抑制拒绝上表现与经典方法相当
- 几何分析揭示概念缺失与对立概念编码方式不同
Arditi等(2024)表明,安全微调聊天模型的拒绝行为由残差流中单一线性方向所调控,可通过有害与无害激活的差异均值(DiM)恢复。本文在五款开源聊天模型上比较了基于DiM的干预(激活添加与方向消融)与两种来自迭代零空间投影(INLP)的方法——零空间投影与反事实翻转,探究INLP是否可匹配DiM的拒绝对齐能力,以及其更丰富的参数化能否带来更可调节的干预效果。结果表明,反事实翻转在抑制拒绝方面与DiM方向消融性能相当,而零空间投影始终较弱。将INLP限制在提取子空间的主方向时,几乎保持原有抑制效果且困惑度接近基线,展现出可调能力。几何分析显示,两种INLP干预落在激活空间中质不同的区域:零空间投影使变换后激活位于有害与无害聚类之间,而反事实翻转则将其移入相反聚类,暗示模型对概念缺失的编码方式不同于其对立面,这一现象值得未来深入研究。
原文摘要 · Abstract (English)
Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recoverable by a difference-in-means (DiM) of harmful and harmless activations. We compare DiM-based interventions (activation addition and directional ablation) with two interventions derived from Iterative Nullspace Projection (INLP) -- nullspace projection and counterfactual flipping -- on five open-weight chat models, asking whether INLP can match DiM at steering refusal and whether its richer parameterisation yields more tweakable interventions. INLP counterfactual flipping is competitive with DiM directional ablation on refusal suppression, while nullspace projection is consistently weaker. Restricting INLP to the leading directions of the extracted subspace preserves most of the suppression effect at near-baseline perplexity, giving a tunable capability. Geometrically, the two INLP interventions land in qualitatively different regions of activation space: nullspace projection collapses transformed activations \emph{between} the harmful and harmless clusters, while counterfactual flipping moves them into the opposite cluster, suggesting that the model encodes the absence of a concept differently from its opposite -- an intriguing distinction that warrants further investigation in future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。