固定大模型等价于有限状态机,演化序列具超图灵计算能力。
Large Language Models and the Extended Church-Turing Thesis
- 固定LLM本质是大型确定性有限状态转换器
- 演化中的LLM序列可模拟带辅助信息的交互图灵机
- 揭示知识生成非算法过程,适合理论计算研究者
扩展邱奇-图灵论题(ECTT)认为所有有效信息处理均可由带辅助信息的交互图灵机描述。当代大语言模型(LLMs)是否符合这一论题?本文从可计算性与计算复杂度理论视角出发,结合自动机理论,揭示了若干基础结论:首先,任何固定(非自适应)的LLM在计算上等价于一个可能非常庞大的确定性有限状态转换器,这刻画了LLM的基础计算层级;其次,我们证明了演化中的LLM序列可模拟空间有界的图灵机。进一步地,我们发现演化序列的LLM在计算上等价于带辅助信息的交互图灵机,验证了其对ECTT的适用性。从可计算性角度看,这表明此类演化序列具备超图灵计算能力。因此,在我们的计算模型中,知识生成本质上是非算法性的,由演化中的LLM序列实现。最后,本文讨论了这些发现对相关学科与哲学的启示。
原文摘要 · Abstract (English)
The Extended Church-Turing Thesis (ECTT) posits that all effective information processing, including unbounded and non-uniform interactive computations, can be described in terms of interactive Turing machines with advice. Does this assertion also apply to the abilities of contemporary large language models (LLMs)? From a broader perspective, this question calls for an investigation of the computational power of LLMs by the classical means of computability and computational complexity theory, especially the theory of automata. Along these lines, we establish a number of fundamental results. Firstly, we argue that any fixed (non-adaptive) LLM is computationally equivalent to a, possibly very large, deterministic finite-state transducer. This characterizes the base level of LLMs. We extend this to a key result concerning the simulation of space-bounded Turing machines by LLMs. Secondly, we show that lineages of evolving LLMs are computationally equivalent to interactive Turing machines with advice. The latter finding confirms the validity of the ECTT for lineages of LLMs. From a computability viewpoint, it also suggests that lineages of LLMs possess super-Turing computational power. Consequently, in our computational model knowledge generation is in general a non-algorithmic process realized by lineages of LLMs. Finally, we discuss the merits of our findings in the broader context of several related disciplines and philosophies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。