用图灵机模拟思维链,让大模型能解更长序列问题。
The Imitation Game: Turing Machine Imitator is Length Generalizable Reasoner
- 用程序生成模仿图灵机运行的思维链数据,拆解推理步骤
- 在8类18个任务上,70亿参数模型长度泛化能力显著提升
- 揭示模型注意力层出现读写行为,验证图灵机制关键作用
长度泛化——即处理比训练时更长序列的能力——是基于Transformer的大语言模型的核心挑战。现有研究多聚焦于算术与符号操作等特定任务的数据驱动方法,但泛化能力有限。本文从可计算推理问题出发,提出图灵机模仿学习(TAIL)框架,通过程序生成模仿图灵机执行过程的思维链数据,将推理步骤线性展开为原子状态以缓解捷径学习,并引入显式记忆访问机制降低基础操作中动态与长距离数据访问难度。为验证其有效性,构建了一个涵盖8类算法、18个任务的合成数据集。仅使用合成数据,未加额外技巧,TAIL显著提升了Qwen2.5-7B在各类任务上的长度泛化性能,超越先前方法及DeepSeek-R1。实验表明,图灵机的核心概念而非思维模式才是实现长度泛化的关键,模型在注意力层展现出与图灵机一致的读写行为。该工作为大模型从合成数据学习推理提供了新方向。
原文摘要 · Abstract (English)
Length generalization, the ability to solve problems of longer sequences than those observed during training, poses a core challenge of Transformer-based large language models (LLM). Although existing studies have predominantly focused on data-driven approaches for arithmetic operations and symbolic manipulation tasks, these approaches tend to be task-specific with limited overall performance. To pursue a more general solution, this paper focuses on a broader case of reasoning problems that are computable, i.e., problems that algorithms can solve, thus can be solved by the Turing Machine. From this perspective, this paper proposes Turing MAchine Imitation Learning (TAIL) to improve the length generalization ability of LLMs. TAIL synthesizes chain-of-thoughts (CoT) data that imitate the execution process of a Turing Machine by computer programs, which linearly expands the reasoning steps into atomic states to alleviate shortcut learning and explicit memory fetch mechanism to reduce the difficulties of dynamic and long-range data access in elementary operations. To validate the reliability and universality of TAIL, we construct a challenging synthetic dataset covering 8 classes of algorithms and 18 tasks. Without bells and whistles, TAIL significantly improves the length generalization ability as well as the performance of Qwen2.5-7B on various tasks using only synthetic data, surpassing previous methods and DeepSeek-R1. The experimental results reveal that the key concepts in the Turing Machine, instead of the thinking styles, are indispensable for TAIL for length generalization, through which the model exhibits read-and-write behaviors consistent with the properties of the Turing Machine in their attention layers. This work provides a promising direction for future research in the learning of LLM reasoning from synthetic data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。