arXiv:2603.21396cs.LG2026-03被引 4

大模型能察觉概念注入并识别内容,且不会误判。

Mechanisms of Introspective Awareness

  • 发现模型通过两阶段电路检测注入信号,早期特征监测扰动,抑制默认负面响应。
  • 检测率中等但零误报,且仅在偏好优化后出现,基础模型无此能力。
  • 该能力可被显著增强,删除拒绝方向后检测率提升53%。

近期研究发现,大型语言模型在残差流中检测到注入的引导向量时,能够识别出被注入的概念——这一现象称为“内省意识”。本文研究了开放权重模型中该能力的机制。首先,该行为具有鲁棒性:模型在多种提示和对话格式下以中等成功率检测到注入的引导向量,且假阳性率为0%。值得注意的是,这种能力仅在后训练阶段出现;我们发现偏好优化算法(如DPO)可引发该能力,而标准监督微调则不能。我们提供证据表明,检测无法由某些引导向量与促进肯定回答的方向之间的简单线性关联解释。我们追踪到检测机制源于一个两阶段电路:早期注入层中的“证据载体”特征沿不同方向单调检测扰动,并抑制下游实施默认负面响应的“门控”特征。该电路在基础模型中不存在,且对拒绝机制消融具有鲁棒性。概念识别依赖于后期层中基本独立的机制,与检测机制重叠度很低。最后,我们发现内省能力被显著低估:消除拒绝方向可使检测率提高+53%,训练偏见向量可在保留外概念上提升+75%检测率,且未明显增加假阳性。结果表明,该内省意识具备鲁棒性和非平凡机制,未来模型中可进一步放大。代码已开源:https://github.com/safety-research/introspection-mechanisms。

原文摘要 · Abstract (English)

Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness." We investigate the mechanisms underlying this capability in open-weights models. First, we find that it is behaviorally robust: models detect injected steering vectors at moderate rates with 0% false positives across diverse prompts and dialogue formats. Notably, this capability emerges specifically from post-training; we show that preference optimization algorithms like DPO can elicit it, but standard supervised finetuning does not. We provide evidence that detection cannot be explained by simple linear association between certain steering vectors and directions promoting affirmative responses. We trace the detection mechanism to a two-stage circuit in which "evidence carrier" features in early post-injection layers detect perturbations monotonically along diverse directions, suppressing downstream "gate" features that implement a default negative response. This circuit is absent in base models and robust to refusal ablation. Identification of injected concepts relies on largely distinct later-layer mechanisms that only weakly overlap with those involved in detection. Finally, we show that introspective capability is substantially underelicited: ablating refusal directions improves detection by +53%, and a trained bias vector improves it by +75% on held-out concepts, both without meaningfully increasing false positives. Our results suggest that this introspective awareness of injected concepts is robust and mechanistically nontrivial, and could be substantially amplified in future models. Code: https://github.com/safety-research/introspection-mechanisms.

大模型内省机制安全检测因果分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。