通过电路分析揭示大模型处理逻辑推理的隐藏机制
A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning
- 用因果中介分析定位模型推理路径与关键组件
- 发现稀疏电路可分解为四个功能模块，逐层执行逻辑运算
- 三款不同规模模型存在相似但不完全相同的推理结构
由于现代大型语言模型（LLMs）的规模和复杂性，理解其解决推理问题的内在机制极具挑战。例如，推理过程是否局限于网络特定部分？是否将问题分解为模块化步骤并随网络深度逐步执行？为更深入理解 LLMs 的推理能力，我们研究了一个最小化的命题逻辑问题，该问题需要结合多个事实才能得出结论。在 Mistral 和 Gemma 模型（最大达 270 亿参数）上进行分析，揭示了模型解决此类逻辑问题的核心组件。从机制可解释性视角出发，我们采用因果中介分析（causal mediation analysis）揭示了模型推理过程中的路径与组件。进一步分析不同层中注意力头的功能，不仅发现一个计算答案的稀疏电路，还将其分解为具有四种不同且模块化用途的子电路。最终发现，Mistral-7B、Gemma-2-9B 和 Gemma-2-27B 三种模型均包含类似但不完全相同的推理机制。
原文摘要 · Abstract (English)
Due to the size and complexity of modern large language models (LLMs), it has proven challenging to uncover the underlying mechanisms that models use to solve reasoning problems. For instance, is their reasoning for a specific problem localized to certain parts of the network? Do they break down the reasoning problem into modular components that are then executed as sequential steps as we go deeper in the model? To better understand the reasoning capability of LLMs, we study a minimal propositional logic problem that requires combining multiple facts to arrive at a solution. By studying this problem on Mistral and Gemma models, up to 27B parameters, we illuminate the core components the models use to solve such logic problems. From a mechanistic interpretability point of view, we use causal mediation analysis to uncover the pathways and components of the LLMs' reasoning processes. Then, we offer fine-grained insights into the functions of attention heads in different layers. We not only find a sparse circuit that computes the answer, but we decompose it into sub-circuits that have four distinct and modular uses. Finally, we reveal that three distinct models -- Mistral-7B, Gemma-2-9B and Gemma-2-27B -- contain analogous but not identical mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。