模型提前感知对话伙伴专家度，但延迟才使用该信息。
Encoded Early, Used Late: Where Transformers Begin to Act on an Inferred Partner's Expertise
- 在早期层就能解码对话伙伴的专家度，但其影响延迟到中后期才显现。
- 峰值可解码层注入差异几乎不影响最终输出，而中期后注入则几乎完全传递。
- 适用于研究模型推理机制或可控对话系统的设计者。
Transformer 可在残差流中较早地线性解码某属性，而该属性尚未影响输出。本文研究这一现象是否适用于需逐步推断的属性——对话伙伴的专家度。基于 ExpertCollab 数据集（四类专家水平的多轮研究规划对话），发现伙伴专家度在早期层最易解码，至网络中点前已降至接近随机水平。反事实补丁实验显示：在峰值可解码层注入专家度差异几乎不改变固定晚层读出结果，而在中点后注入则几乎完全传播，差距超过一个数量级。内容匹配的随机对照与无探针诊断均将转变点定位在早期层，静态属性则全程可解码。因此，推断型关系属性在因果生效前即已被充分表示，限制了行为调控的干预时机。研究以单一模型在合成数据集上完成初步验证。
原文摘要 · Abstract (English)
A transformer can make an attribute linearly decodable in its residual stream at a depth where that attribute does not yet influence the output. This gap between where information is readable and where it is used has been shown for attributes stated directly in the input. We ask whether it also holds for an attribute the model must infer gradually over a conversation, namely how expert its dialogue partner is. Using ExpertCollab, a corpus of multi-turn research-planning dialogues between model-played personas at four expertise levels, we find that partner expertise is most decodable in the early layers and falls to near chance before the midpoint of the network. Counterfactual patching shows that injecting the expertise difference at the layer of peak decodability barely changes a fixed late-layer readout, whereas the same difference injected past the midpoint propagates almost completely, a separation of more than an order of magnitude. A content-matched random control and a probe-free diagnostic place the transition at the same early layer, and a statically specified control attribute stays decodable throughout. An inferred relational attribute is therefore represented well before it becomes causally active, which bounds where any attempt to read out or steer partner-conditioned behavior must intervene. We use one model on a synthetic corpus as an initial demonstration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。