语言模型的推理是统计模式匹配，而非真正逻辑推理。
On the Notion that Language Models Reason
- 将语言模型视为隐式马尔可夫核，通过上下文预测词元分布。
- 推理类输出源于训练数据中的统计规律，非显式逻辑推导。
- 适合关注模型本质与认知误解的研究者阅读。
语言模型被称作具备推理能力，但这究竟意味着什么？本文评估了自然语言处理领域中关于推理的定义及其使用方式，指出这些定义与语言模型的训练机制、信息处理方式及新词元生成过程不一致。为阐明这种不一致性，本文假设基于Transformer的语言模型实现了一种隐式有限阶马尔可夫核，将上下文映射为条件词元分布。在此视角下，看似推理的输出仅对应于学习到的核中的统计规律与近似统计不变性，而非显式逻辑机制的实现。这一观点支持语言模型是‘统计模式匹配器’而非真正推理者的论断，有助于理解为何语言模型会产生推理类输出却无逻辑一致性保障。这一区分对评估语言模型的信念不确定性至关重要。本文呼吁重新审视自然语言处理研究中对系统计算过程的描述方式。
原文摘要 · Abstract (English)
Language models (LMs) are said to be exhibiting reasoning, but what does this entail? We assess definitions of reasoning and how key papers in the field of natural language processing (NLP) use the notion and argue that the definitions provided are not consistent with how LMs are trained, process information, and generate new tokens. To illustrate this incommensurability we assume the view that transformer-based LMs implement an \textit{implicit} finite-order Markov kernel mapping contexts to conditional token distributions. In this view, reasoning-like outputs correspond to statistical regularities and approximate statistical invariances in the learned kernel rather than the implementation of explicit logical mechanisms. This view is illustrative of the claim that LMs are "statistical pattern matchers"" and not genuine reasoners and provides a perspective that clarifies why reasoning-like outputs arise in LMs without any guarantees of logical consistency. This distinction is fundamental to how epistemic uncertainty is evaluated in LMs. We invite a discussion on the importance of how the computational processes of the systems we build and analyze in NLP research are described.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。