解析大模型如何通过注意力机制实现上下文检索增强。
The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation
- 通过归因分析定位专门处理指令与信息检索的注意力头。
- 发现模型中存储实体关系知识的参数化注意力头。
- 可追踪推理中的知识来源，提升模型透明性与安全性。
大型语言模型能够通过检索增强实现上下文学习，获取训练数据之外的外部知识。尽管前景广阔，其内在机制仍不清晰。本文将提示视为信息组件的组合，提出一种基于归因的方法，识别出理解指令并检索相关上下文信息的上下文注意力头，以及存储实体间关系知识的参数化注意力头。通过提取功能向量并修改注意力权重，揭示其对答案生成的影响。最终，利用这些洞察追踪推理过程中使用的知识来源，为构建更安全、更透明的语言模型铺平道路。
原文摘要 · Abstract (English)
Large language models are able to exploit in-context learning to access external knowledge beyond their training data through retrieval-augmentation. While promising, its inner workings remain unclear. In this work, we shed light on the mechanism of in-context retrieval augmentation for question answering by viewing a prompt as a composition of informational components. We propose an attribution-based method to identify specialized attention heads, revealing in-context heads that comprehend instructions and retrieve relevant contextual information, and parametric heads that store entities' relational knowledge. To better understand their roles, we extract function vectors and modify their attention weights to show how they can influence the answer generation process. Finally, we leverage the gained insights to trace the sources of knowledge used during inference, paving the way towards more safe and transparent language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。