推理模型能高效模拟高级算法,仅需对数级额外开销。
Efficiently Representing Algorithms With Chain-of-Thought Transformers
- 用链式思维变压器模拟字随机存取机算法,提升计算效率。
- 可在多项式对数时间内完成排序与最短路径等经典算法。
- 适用于连续推理和混合架构,适合实际部署的推理模型设计。
推理型语言模型通过输出一系列思考步骤来生成答案,其理论基础在于链式思维(CoT)Transformer可模拟图灵机,从而实现任意计算。然而图灵机抽象层级低,不便于描述实际算法。相比之下,字随机存取机(Word RAM)模型以单位时间操作 $igO("log n$) 位字为前提,更贴近真实算法设计。因此问题在于:CoT Transformer能否高效模拟 Word RAM 算法?例如,能否在 $igO(n "log n$) 步内排序 $n$ 个元素,或在 $igO(E + V "log V$) 步内运行戴克斯特拉算法?我们给出肯定回答,仅需多项式对数级额外开销。首先在有限精度、多项式对数宽度及右端唯一硬注意力条件下证明该结果,随后推广至更实用场景:连续式链式思维(向量形式推理)与基于线性RNN的混合架构。三类设定下,CoT均能以多对数开销高效模拟任何 Word RAM 算法。若指令集为“扁平化”且无乘法,则开销降至 $igO("log^2 n$),甚至仅为 $igO("log n$),远优于已知图灵机模拟所需的二次开销。
原文摘要 · Abstract (English)
The increasing popularity of \emph{reasoning} models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by theoretical results showing that chain-of-thought (CoT) transformers can simulate Turing machines, and thus perform arbitrary computation. However, the Turing machine, while suitable for complexity-theoretic analysis, is not convenient, intuitive, or efficient for discussing algorithms. Algorithms are typically designed and analyzed at a higher level of abstraction, captured by the \emph{Word RAM} model with random-access memory and unit-cost operations on $\bigO(\log n)$-bit words. As a result, Word RAM algorithms can be substantially more efficient than their Turing machine counterparts, raising the question: \emph{Can CoT transformers efficiently simulate Word RAM algorithms?} For instance, can they sort $n$ items in $\bigO(n \log n)$ steps or run Dijkstra's algorithm in $\bigO(E + V \log V)$ steps? We answer affirmatively, up to poly-logarithmic overhead. We first establish this for finite-precision transformers with poly-logarithmic width and rightmost unique hard attention, then strengthen the result to two more practical settings with finite width and log-precision: \emph{continuous} CoT, where reasoning takes the form of vectors rather than tokens, and a \emph{hybrid} architecture in which transformer layers sit atop a recurrent (linear RNN) layer. In all three cases, we find that CoT \emph{can} efficiently simulate any Word RAM algorithm with only a poly-logarithmic overhead in $n$. This overhead reduces to log-square when the Word RAM has a ``flat'' instruction set, and only logarithmic for multiplication-free flat instructions -- in stark contrast to known CoT simulations of Turing machines, which require quadratic overhead over Word RAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。