arXiv:2507.00239cs.CLcs.AI2025-07被引 3

发现对齐模型拒绝的知识仍可线性解码,且影响下游行为。

Linearly Decoding Refused Knowledge in Aligned Language Models

  • 用线性探针从模型隐藏状态中解码越狱提示获取的信息。
  • 平均国家智商等数据的预测相关系数超0.8,表明信息高度可解码。
  • 基础模型的表征能迁移至对齐模型,揭示被抑制的潜在知识。

主流语言模型通过微调与强化学习进行指令微调和对齐,导致其拒绝用户提出的有害请求。然而,越狱提示常可绕过这些拒绝机制,诱使模型生成有害回应。本文研究越狱提示所访问信息是否可通过训练于模型隐藏状态的线性探针进行解码。结果表明,大量原本被拒绝的信息具有高度线性可解码性。例如,在多个模型中,越狱后模型对各国平均智商的响应可用线性探针预测,皮尔逊相关系数超过0.8。令人意外的是,基于基础模型(不拒绝)训练的探针有时可迁移到指令对齐版本,并揭示越狱生成方式解码的信息,表明许多被拒绝属性的内部表征在指令微调过程中持续保留。更重要的是,这些信息并非“残留物”,而是被模型主动使用:探针预测值与模型生成的成对比较结果存在相关性,说明解码信息与被抑制的生成行为一致,可能在其他下游任务中以更隐晦方式表现。总体而言,指令微调并未彻底消除或重定位有害信息,仅抑制其直接表达,使其仍在线性上可访问,并间接影响下游行为。

原文摘要 · Abstract (English)

Most commonly used language models (LMs) are instruction-tuned and aligned using a combination of fine-tuning and reinforcement learning, causing them to refuse users requests deemed harmful by the model. However, jailbreak prompts can often bypass these refusal mechanisms and elicit harmful responses. In this work, we study the extent to which information accessed via jailbreak prompts is decodable using linear probes trained on LM hidden states. We show that a great deal of initially refused information is linearly decodable. For example, across models, the response of a jailbroken LM for the average IQ of a country can be predicted by a linear probe with Pearson correlations exceeding $0.8$. Surprisingly, we find that probes trained on base models (which do not refuse) sometimes transfer to their instruction-tuned versions and are capable of revealing information that jailbreaks decode generatively, suggesting that the internal representations of many refused properties persist from base LMs through instruction-tuning. Importantly, we show that this information is not merely "leftover" in instruction-tuned models, but is actively used by them: we find that probe-predicted values correlate with LM generated pairwise comparisons, indicating that the information decoded by our probes align with suppressed generative behavior that may be expressed more subtly in other downstream tasks. Overall, our results suggest that instruction-tuning does not wholly eliminate or even relocate harmful information in representation space-they merely suppress its direct expression, leaving it both linearly accessible and indirectly influential in downstream behavior.

语言模型越狱攻击知识解码表征分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。