arXiv:2509.09708cs.CLcs.AI2025-09AAAI被引 13

揭秘大模型拒绝有害指令的内在机制,找到可触发破解的关键特征。

Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal

  • 通过稀疏自编码器分析模型内部激活,定位拒绝行为的潜在方向。
  • 发现一组关键特征,其移除可使模型从拒绝转为服从,实现破解。
  • 揭示冗余特征的存在,为安全干预提供可操作的解释路径。

指令微调的大语言模型在面对有害提示时表现出拒绝行为,但其内部机理尚不明确。本研究针对两个公开模型(Gemma-2-2B-IT 和 LLaMA-3.1-8B-IT),利用残差流激活训练的稀疏自编码器(SAE)进行分析。针对有害提示,通过搜索 SAE 隐空间中能导致模型从拒绝转为服从的特征集合,验证其因果影响并实现越狱。该流程分为三阶段:(1) 拒绝方向识别:寻找中介拒绝的方向并收集邻近特征;(2) 贪心筛选:精简至最小特征集;(3) 交互发现:使用因子分解机(FM)捕捉剩余活跃特征间的非线性交互。该方法揭示了大量越狱关键特征,深入揭示了拒绝行为的机制基础。此外,还发现部分特征仅在早期特征被抑制时才激活,表明存在冗余结构。结果表明,通过操控可解释的隐空间,可实现对安全行为的细粒度审计与靶向干预。

原文摘要 · Abstract (English)

Refusal on harmful prompts is a key safety behaviour in instruction-tuned large language models (LLMs), yet the internal causes of this behaviour remain poorly understood. We study two public instruction-tuned models, Gemma-2-2B-IT and LLaMA-3.1-8B-IT, using sparse autoencoders (SAEs) trained on residual-stream activations. Given a harmful prompt, we search the SAE latent space for feature sets whose ablation flips the model from refusal to compliance, demonstrating causal influence and creating a jailbreak. Our search proceeds in three stages: (1) Refusal Direction: find a refusal-mediating direction and collect SAE features near that direction; (2) Greedy Filtering: prune to a minimal set; and (3) Interaction Discovery: fit a factorization machine (FM) that captures nonlinear interactions among the remaining active features and the minimal set. This pipeline yields a broad set of jailbreak-critical features, offering insight into the mechanistic basis of refusal. Moreover, we find evidence of redundant features that remain dormant unless earlier features are suppressed. Our findings highlight the potential for fine-grained auditing and targeted intervention in safety behaviours by manipulating the interpretable latent space.

大模型安全可解释性越狱检测隐空间分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。