通过分析模型权重，揭示语言模型第一层注意力的解码机制。
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
- 用权重分解法解析GPT-2首层注意力结构
- 发现近邻词偏好与解码行为的权重依据
- 无需推理即可理解模型早期语义构建
根据推理阶段假说，语言模型的早期层将子词分词输入映射为更具语义意义的表示，形成模型的“内部词汇”。以往对这一解码阶段的分析主要依赖探测和干预方法，如路径修补，需选定特定输入、部分组件并观察模型行为变化。本文表明，无需任何模型推理步骤，仅通过分析模型权重即可理解解码阶段的多个关键方面。我们提出对GPT-2首层注意力的解析分解，得到可解释的项，量化位置相关、词元相关及混合效应的相对贡献。聚焦这些分解项，我们发现了权重层面解释注意力偏向邻近词及解码行为的机制。
原文摘要 · Abstract (English)
According to the stages-of-inference hypothesis, early layers of language models map their subword-tokenized input, which does not necessarily correspond to a linguistically meaningful segmentation, to more meaningful representations that form the model's "inner vocabulary". Prior analysis of this detokenization stage has predominantly relied on probing and interventions such as path patching, which involve selecting particular inputs, choosing a subset of components that will be patched, and then observing changes in model behavior. Here, we show that several important aspects of the detokenization stage can be understood purely by analyzing model weights, without performing any model inference steps. Specifically, we introduce an analytical decomposition of first-layer attention in GPT-2. Our decomposition yields interpretable terms that quantify the relative contributions of position-related, token-related, and mixed effects. By focusing on terms in this decomposition, we discover weight-based explanations of attention bias toward close tokens and attention for detokenization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。