揭示大模型进行命题逻辑推理的内部计算机制
Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models
- 发现四类协同运作的计算机制，实现分阶段推理
- 在11类逻辑规则中均表现出稳定结构化策略
- 适用于不同规模模型，适合研究模型可解释性
理解大语言模型（LLMs）内部如何执行逻辑推理仍是基础性挑战。现有机制研究多聚焦特定任务回路，却未回答模型采用何种计算策略进行命题推理。本文通过全面分析Qwen3（8B和14B）在PropLogic-MI上的表现，该数据集覆盖11类命题逻辑规则，涵盖单跳与双跳推理。不问“哪些组件必要”，而关注“模型如何组织计算”。分析揭示一个由四类互锁机制构成的统一计算架构：分阶段计算（逐层处理）、信息传输（边界标记处汇聚信息流）、事实回溯（持续重访原始事实）、专用注意力头（功能各异的头类型）。这些机制在模型规模、规则类型和推理深度间具有泛化能力，为大模型采用结构化计算策略进行逻辑推理提供了机制证据。
原文摘要 · Abstract (English)
Understanding how Large Language Models (LLMs) perform logical reasoning internally remains a fundamental challenge. While prior mechanistic studies focus on identifying taskspecific circuits, they leave open the question of what computational strategies LLMs employ for propositional reasoning. We address this gap through comprehensive analysis of Qwen3 (8B and 14B) on PropLogic-MI, a controlled dataset spanning 11 propositional logic rule categories across one-hop and two-hop reasoning. Rather than asking ''which components are necessary,'' we ask ''how does the model organize computation?'' Our analysis reveals a coherent computational architecture comprising four interlocking mechanisms: Staged Computation (layer-wise processing phases), Information Transmission (information flow aggregation at boundary tokens), Fact Retrospection (persistent re-access of source facts), and Specialized Attention Heads (functionally distinct head types). These mechanisms generalize across model scales, rule types, and reasoning depths, providing mechanistic evidence that LLMs employ structured computational strategies for logical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。