大模型拼写字符时依赖深层网络重建，而非直接读取嵌入层。
Spelling-out is not Straightforward: LLMs' Capability of Tokenization from Token to Characters
- 嵌入层未完全编码字符信息，尤其在首字符之后。
- 中高层变换器层实现字符知识的突破性重构。
- 适合研究模型内部表征机制的研究者阅读。
大型语言模型（LLMs）能以高准确率逐字符拼写令牌，但在更复杂的字符级任务上表现不佳，例如识别令牌中的组合子成分。本文研究了LLMs在拼写过程中如何内部表示和利用字符级信息。分析表明，尽管对人类而言拼写是简单任务，但对LLMs并非直截了当。具体而言，我们发现嵌入层并未完全编码字符级信息，尤其是首字符之后的部分。因此，LLMs依赖中间和高层变换器层来重构字符级知识，并在该过程中观察到明显的“突破”现象。通过三种互补分析——探测分类器、知识神经元识别和注意力权重检查——验证了这一机制。
原文摘要 · Abstract (English)
Large language models (LLMs) can spell out tokens character by character with high accuracy, yet they struggle with more complex character-level tasks, such as identifying compositional subcomponents within tokens. In this work, we investigate how LLMs internally represent and utilize character-level information during the spelling-out process. Our analysis reveals that, although spelling out is a simple task for humans, it is not handled in a straightforward manner by LLMs. Specifically, we show that the embedding layer does not fully encode character-level information, particularly beyond the first character. As a result, LLMs rely on intermediate and higher Transformer layers to reconstruct character-level knowledge, where we observe a distinct "breakthrough" in their spelling behavior. We validate this mechanism through three complementary analyses: probing classifiers, identification of knowledge neurons, and inspection of attention weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。