发现实体与关系知识存储方式不同，关系信息藏在注意力模块中。
Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in Language Models
- 通过知识编辑对比实体和关系知识的可迁移性
- 关系知识显著编码在注意力模块而非仅存于MLP层
- 揭示大模型知识存储的复杂性，适合模型可解释性研究者
大型语言模型在多种自然语言处理任务中表现出色，并已证实其知识可定位到特定参数，如中间层的MLP权重。本研究通过知识编辑方法，探究实体与关系知识之间的差异。结果表明，实体知识与关系知识无法直接相互转换或映射，这一发现出乎意料，因为逻辑上修改同一知识三元组中的实体或关系应产生相似效果。为进一步阐明二者差异，我们采用因果分析探究关系知识在预训练模型中的存储机制。实验显示，关系知识不仅存在于MLP权重中，还显著编码在注意力模块中。该发现揭示了语言模型知识存储的多面性，强调了在模型中操控特定类型知识的复杂性。
原文摘要 · Abstract (English)
Large language models encapsulate knowledge and have demonstrated superior performance on various natural language processing tasks. Recent studies have localized this knowledge to specific model parameters, such as the MLP weights in intermediate layers. This study investigates the differences between entity and relational knowledge through knowledge editing. Our findings reveal that entity and relational knowledge cannot be directly transferred or mapped to each other. This result is unexpected, as logically, modifying the entity or the relation within the same knowledge triplet should yield equivalent outcomes. To further elucidate the differences between entity and relational knowledge, we employ causal analysis to investigate how relational knowledge is stored in pre-trained models. Contrary to prior research suggesting that knowledge is stored in MLP weights, our experiments demonstrate that relational knowledge is also significantly encoded in attention modules. This insight highlights the multifaceted nature of knowledge storage in language models, underscoring the complexity of manipulating specific types of knowledge within these models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。