揭秘大模型如何把子词拼成单词，发现早期层有固定两阶段机制。
Inside the LLM Word Factory
- 通过激活修补实验，发现单词重组分两阶段：注意力传信号，MLP融合嵌入
- 在Llama2-7B中，该过程集中在第1层，不同位置编码模型跨度1-10层
- 仅用早期激活就能以0.94-0.97的准确率判断单词重组是否成功
Transformer语言模型以子词片段为输入，但自然语言语义通常依赖词级概念。解码化（detokenization）是模型将子词聚合为词级表示的过程。已有研究指出该过程主要发生在早期到中期层，但具体机制尚不明确。本文通过控制性配对实验中的激活修补技术，深入分析解码化机制，发现Llama2-7B中的英语解码化遵循两阶段过程：注意力模块从非末尾子词传递特定于词的信号，必要时通过序列接力完成；而MLP模块则将该信号与局部嵌入组合。这一两阶段结构在来自八个模型家族的十二个模型中具有普适性，但其作用深度受位置编码类型影响：基于RoPE的模型在1至5层内完成，而使用学习型绝对编码的模型则需5至10层。最后，我们提出一种基于早期层激活的探测器，仅凭早期激活即可判断解码化成功与否，其性能在0.94至0.97之间，取决于上下文长度。
原文摘要 · Abstract (English)
Transformer language models process input provided as subword fragments, but natural language semantics usually rely on word-level concepts. Detokenization is the process where models reconcile these two facts, aggregating subwords into word-level representations through their computation. Prior work has found that this takes place mostly in early-to-middle layers, but so far the exact mechanics of the process have not been pinned down. We venture deep into detokenization using activation patching in controlled paired experiments that isolate the contribution of different model components, localizing English detokenization in Llama2-7B to a two-stage process at Layer 1. Attention transmits a token-specific signal from nonfinal subwords, using sequential relays if necessary, while the MLP composes it with the local embedding. This two-stage structure generalizes to twelve models from eight families, but the depth over which it takes place depends on the flavor of positional encoding: RoPE-based models detokenize over 1 to 5 layers, while learned-absolute models take 5 to 10. Finally, we provide a probe for determining the success of the detokenization process based on early-layer activations alone, performing at 0.94-0.97 AUROC depending on the amount of context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。