揭示大模型推理过程的内在几何结构，发现中层表示最清晰区分不同推理操作。
Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
- 通过隐藏层表征分析推理操作的几何分离性。
- 中层表征对推理操作的区分度最高，且不受词法位置干扰。
- 适合关注模型内部机制与可解释性的研究人员。
大语言模型中的推理涉及问题定义、目标分解和推导等多种功能操作。尽管这些操作在文本中明显可辨，但其在表征空间中的几何组织方式仍不明确。本文研究不同推理操作是否在隐藏表征中呈现对应几何结构。结果表明，推理操作在保留表征中具有可分性，且中层表现最佳；该结构非由词法或位置因素导致。跨层分析显示，词粒度的推理对齐逐渐分布于更长片段，相同表面词在不同推理上下文中表征不同。注意力掩码干预进一步表明，块起始处的操作对齐表征依赖于前序推理上下文。本研究证明语言模型在语言推理表达与其内部几何结构间保持表征对应。代码与项目材料见 https://github.com/naver-ai/beneath-cot。
原文摘要 · Abstract (English)
Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To this end, we investigate whether distinct reasoning operations exhibit corresponding geometric structure in hidden representations. We find that operations are separable in held-out representations, with separability peaking in middle layers, and verify that this structure is not explained by lexical or positional confounds. Across layers, token-wise operation-alignment becomes more distributed over spans, while identical surface tokens are represented differently depending on the operation of its surrounding chunk. Attention-masking interventions further show that operation-aligned representations at chunk onset depend on preceding reasoning context. Consequently, our work demonstrates that language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures. Code and project materials are available at https://github.com/naver-ai/beneath-cot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。