研究大模型如何组合技能完成新任务，发现模块化机制可自然催生泛化能力。
Assign and Add: A Mechanistic Study of Compositional Arithmetic
- 用变量赋值与模加法构造可控实验，验证模型组合能力
- 小模型能泛化到训练中未见的变量与数字组合
- 模块化添加层在直接或间接输入下行为一致，适合研究模型机制
大型语言模型能够组合已有技能以完成训练中未见过的复杂任务，但其组合机制仍不清晰。本文通过变量赋值与模加法的受控实验，研究Transformer中的组合泛化机制。将训练数据划分为互斥集合后，小型Transformer展现出对未见变量与数字组合的泛化能力。机械分析表明，无论输入是直接还是通过变量赋值间接给出，模型均使用相同的‘模加法’MLP模块。经验性训练动态分析揭示三个阶段：先学模加法，再学变量赋值结构，最后进入泛化难序列的精炼阶段。最后，提出理论框架解释组合性如何从训练动态中涌现。结果表明，组合泛化可能是Transformer内部机制组合性的自然结果。
原文摘要 · Abstract (English)
Large language models are able to compose skills in order to perform complex tasks, many of which might not have been seen during training. The details of how exactly this composition occurs remain elusive. In this paper, we study a mechanism for compositional generalization in transformers by considering a simple controlled setting involving variable assignment and modular addition. By partitioning our training data into disjoint sets, we observe that small transformers are able to generalize to previously unseen combinations of variables and numbers. Our mechanistic analysis shows that the same ``modular addition'' MLP module is used whether the inputs are given directly or indirectly through a separate variable assignment mechanism. We also analyze the training dynamics from an empirical lens, which reveals three phases of learning: first, modular addition is learned, then the structure required for variable assignment, and finally a refinement phase where the model generalizes to some hard sequences not seen in training. Finally, we provide a theoretical framework to explain how compositionality emerges from training dynamics. These results suggest that compositional generalization can be a natural consequence of the compositionality of internal mechanisms in~transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。