研究单层Transformer如何在排序任务中自发形成简化结构。
Structure Development in List-Sorting Transformers
- 模型通过注意力头实现词汇分割与复制抑制，比多头重叠更简单
- 无论是否使用权重衰减，均出现词汇分割现象
- 揭示训练数据如何塑造Transformer内部结构，适合研究模型机制者
我们研究单层仅注意力的Transformer在学习排序数字列表时如何发展出相关结构。训练结束后，模型的注意力头呈现出两种主要模式：词汇分割和复制抑制。这两种模式均比多个注意力头处理重叠数值范围更简单。有趣的是，即使不使用权重衰减（一种常用于促进简化的正则化技术），词汇分割仍存在，支持神经网络自然偏好简单解的论点。我们将复制抑制与GPT-2中的机制关联，并探究其在本模型中的功能作用。基于对模型演进过程的分析，我们识别出训练数据中驱动最终解决方案的关键特征。这为训练数据如何塑造Transformer内部结构提供了具体例证，为未来理解大模型内部结构演化奠定了基础。
原文摘要 · Abstract (English)
We study how a one-layer attention-only transformer develops relevant structures while learning to sort lists of numbers. At the end of training, the model organizes its attention heads in two main modes that we refer to as vocabulary-splitting and copy-suppression. Both represent simpler modes than having multiple heads handle overlapping ranges of numbers. Interestingly, vocabulary-splitting is present regardless of whether we use weight decay, a common regularization technique thought to drive simplification, supporting the thesis that neural networks naturally prefer simpler solutions. We relate copy-suppression to a mechanism in GPT-2 and investigate its functional role in our model. Guided by insights from a developmental analysis of the model, we identify features in the training data that drive the model's final acquired solution. This provides a concrete example of how the training data shape the internal organization of transformers, paving the way for future studies that could help us better understand how LLMs develop their internal structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。