用稀疏自编码器揭示大模型拒绝回答的内在机制
Understanding Refusal in Language Models with Sparse Autoencoders
- 通过稀疏自编码器识别驱动拒绝行为的潜在特征
- 干预这些特征可显著影响生成结果，验证其因果作用
- 适合研究模型安全、对抗攻击与可解释性的研究人员
拒绝是对齐语言模型中的关键安全行为，但其内部机制仍不清晰。本文利用稀疏自编码器对指令微调的大模型中的拒绝行为进行机制研究，识别出因果性介导拒绝的潜在特征。方法应用于两个开源聊天模型，在多个有害数据集上干预相关特征并评估其对生成的影响，验证了其行为效应。该研究实现了对拒绝在激活层面的细粒度分析，解决了上游-下游潜在关系等关键问题，并发现拒绝特征能提升线性探测器在分布外对抗样本上的泛化能力。代码已开源。
原文摘要 · Abstract (English)
Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。