揭示大模型与大脑在计算层级上的三重结构及关键跃迁。
Three tiers of computation in transformers and in brain architectures
- 提出计算能力的三层架构与两种跃迁机制。
- 实证发现模型能力取决于层级跃迁而非单纯规模扩大。
- 适合关注大模型逻辑短板与认知机制的研究者。
人类语言与逻辑能力可在经典的语法-自动机层级中进行计算量化。我们识别出三个层级及对应的两次跃迁，并证明其与基于Transformer的语言模型（LMs）的特定能力存在对应关系。这些涌现能力常被归因于模型规模，但我们表明，决定系统能力的是层级间的跃迁，而非规模本身。人类可轻松处理语言，但需专门训练才能进行算术或逻辑推理；而语言模型虽具备前代系统所无的语言能力，却仍难以处理逻辑任务。本文提出一种新的计算能力基准，对人类和十五个语言模型进行了实证评估，并提供了一个理论严谨的分析框架，以促进对这些关键问题的深入思考。该框架为模型的能力与局限提供了可解释性说明，并为扩展其逻辑能力提供了可操作的洞见。
原文摘要 · Abstract (English)
Human language and logic abilities are computationally quantified within the well-studied grammar-automata hierarchy. We identify three hierarchical tiers and two corresponding transitions and show their correspondence to specific abilities in transformer-based language models (LMs). These emergent abilities have often been described in terms of scaling; we show that it is the transition between tiers, rather than scaled size itself, that determines a system's capabilities. Specifically, humans effortlessly process language yet require critical training to perform arithmetic or logical reasoning tasks; and LMs possess language abilities absent from predecessor systems, yet still struggle with logical processing. We submit a novel benchmark of computational power, provide empirical evaluations of humans and fifteen LMs, and, most significantly, provide a theoretically grounded framework to promote careful thinking about these crucial topics. The resulting principled analyses provide explanatory accounts of the abilities and shortfalls of LMs, and suggest actionable insights into the expansion of their logic abilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。