深度神经网络逻辑推理能力受限于层数，每层仅能处理一层逻辑。
On the Limits of Hierarchically Embedded Logic in Classical Neural Networks
- 将神经网络视为逻辑谓词空间的线性算子，揭示其逻辑表达上限
- 深度不足时无法表征高阶逻辑，如复杂谓词上的计数
- 解释幻觉、重复等现象，指导未来模型架构改进
我们提出一个关于大语言模型推理能力局限性的形式化模型，基于神经网络的深度。通过将神经网络视为逻辑谓词空间上的线性算子，我们证明每一层最多只能编码一层额外的逻辑推理。因此，深度为特定值的神经网络无法忠实表示高一阶逻辑中的谓词，例如在复杂谓词上的简单计数，这表明逻辑表达能力存在严格上界。该结构在分词和嵌入阶段引发非平凡零空间，排除了高阶谓词的可表征性。我们的框架自然解释了幻觉、重复和有限规划等现象，并为高阶逻辑近似如何产生提供了基础。这些结果推动了未来语言模型的架构扩展与可解释性策略。
原文摘要 · Abstract (English)
We propose a formal model of reasoning limitations in large neural net models for language, grounded in the depth of their neural architecture. By treating neural networks as linear operators over logic predicate space we show that each layer can encode at most one additional level of logical reasoning. We prove that a neural network of depth a particular depth cannot faithfully represent predicates in a one higher order logic, such as simple counting over complex predicates, implying a strict upper bound on logical expressiveness. This structure induces a nontrivial null space during tokenization and embedding, excluding higher-order predicates from representability. Our framework offers a natural explanation for phenomena such as hallucination, repetition, and limited planning, while also providing a foundation for understanding how approximations to higher-order logic may emerge. These results motivate architectural extensions and interpretability strategies in future development of language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。