探究大模型如何记忆和提取关系知识，找出最适合作关系分类的隐藏表示。
Tracing Relational Knowledge Recall in Large Language Models
- 通过线性探测分析注意力头与MLP贡献的潜在表征，发现每头注意力对残差流的贡献更优。
- 关系类型越具体、实体连接越紧密，越容易被线性模型准确识别。
- 利用词元级特征归因可深入揭示分类探针的行为机制，适合模型可解释性研究者。
我们研究大语言模型在文本生成过程中如何回忆关系知识，重点是通过线性探测识别适用于关系分类的潜在表征。已有研究揭示了注意力头与MLP如何协同解析主语、谓语和宾语，但尚不清楚哪些表征能支持忠实的线性关系分类，以及为何某些关系类型更容易被线性捕捉。我们系统评估了来自注意力头和MLP贡献的不同潜在表征，发现每个注意力头对残差流的贡献是线性关系分类中表现最强的特征。对训练后探针的特征归因分析，以及不同关系类型的特性表明，探针准确率与关系特异性、实体连通性，以及探针所依赖信号在注意力头间的分布程度密切相关。最后，我们展示了如何利用词元级特征归因进一步揭示探针行为细节。
原文摘要 · Abstract (English)
We study how large language models recall relational knowledge during text generation, with a focus on identifying latent representations suitable for relation classification via linear probes. Prior work shows how attention heads and MLPs interact to resolve subject, predicate, and object, but it remains unclear which representations support faithful linear relation classification and why some relation types are easier to capture linearly than others. We systematically evaluate different latent representations derived from attention head and MLP contributions, showing that per-head attention contributions to the residual stream are comparatively strong features for linear relation classification. Feature attribution analyses of the trained probes, as well as characteristics of the different relation types, reveal clear correlations between probe accuracy and relation specificity, entity connectedness, and how distributed the signal on which the probe relies is across attention heads. Finally, we show how token-level feature attribution of probe predictions can be used to reveal probe behavior in further detail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。