用多带图灵机解析大模型推理失败，定位错误环节。
Understanding LLM Failures: A Multi-Tape Turing Machine Analysis of Systematic Errors in Language Model Reasoning
- 将大模型各组件映射为多带图灵机的独立纸带，精确追踪计算过程。
- 发现分词会掩盖计数任务所需的字符级结构，导致错误。
- 解释思维链提示为何有效，也揭示其内在局限性。
大语言模型在看似简单的任务中仍会出现错误。本文提出一种基于确定性多带图灵机的形式化建模方法，将输入字符、分词、词汇表、模型参数、激活值、概率分布和输出文本等组件分别对应到不同的纸带。该模型可精确定位错误发生的具体处理阶段，例如揭示分词过程会破坏计数任务所需的字符级结构。同时，该框架解释了思维链提示为何能提升性能——通过将计算外化到输出纸带上，但也揭示了其根本局限性。该方法为几何隐喻提供了可验证的替代方案，并在经验缩放定律之外，提供了一种严谨的错误分析范式。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit failure modes on seemingly trivial tasks. We propose a formalisation of LLM interaction using a deterministic multi-tape Turing machine, where each tape represents a distinct component: input characters, tokens, vocabulary, model parameters, activations, probability distributions, and output text. The model enables precise localisation of failure modes to specific pipeline stages, revealing, e.g., how tokenisation obscures character-level structure needed for counting tasks. The model clarifies why techniques like chain-of-thought prompting help, by externalising computation on the output tape, while also revealing their fundamental limitations. This approach provides a rigorous, falsifiable alternative to geometric metaphors and complements empirical scaling laws with principled error analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。