揭示大模型推理时隐藏表示如何动态变化，让抽象概念更清晰。
Fluid Reasoning Representations
- 通过分析推理过程中的表示变化，发现动作与谓词概念逐渐对齐。
- 不同表述下相同概念的表示相似度提升，逼近未混淆的真实概念。
- 适合研究模型内部推理机制的学者，尤其关注认知建模与可解释性。
前沿的大语言模型通过扩展测试时推理解决复杂任务，但其隐藏状态表示在推理过程中如何演变尚不明确。本文提出‘流式推理表征’（Fluid Reasoning Representations, FRR），从表征层面解析模型在自生成推理中如何组织动作与谓词概念。我们在经过混淆的规划、符号及数学任务上进行测试，保持问题结构但替换关键词汇。在开放权重基础模型、指令微调模型和扩展推理模型中，同一动作或谓词的表示随推理推进趋于一致，并趋近于未混淆的真实概念。因果探测显示这些表示影响行为：跨命名引导提升保留准确率，优于高斯与随机对照；符号修复保留更多动作信息，高于随机修复；移除优化方向则降低准确率。结果表明，扩展推理强化了原本存在于基础与指令微调模型中的表示动态。代码已开源。
原文摘要 · Abstract (English)
Frontier large language models increasingly solve complex tasks involving abstract concepts through extended test-time thinking. Yet we lack a mechanistic account of how extended thinking changes hidden-state representations over the course of a reasoning trace. We introduce \textit{Fluid Reasoning Representations} (FRRs), a representation-level account of how LLMs organize action and predicate concepts during self-generated reasoning, and test them on obfuscated planning, symbolic, and mathematical tasks where task-relevant words are replaced while problem structure is preserved. Across open-weight base, instruction-tuned, and extended-thinking LLMs, representations of the same action or predicate become more similar across wordings and move toward the corresponding unobfuscated concepts over the reasoning trace. Causal probes show that these representations affect behavior: cross-naming steering improves held-out accuracy beyond Gaussian and shuffled controls, symbolic patching retains more action information than shuffled patching, and subtracting refined directions degrades accuracy; together, these results suggest that extended thinking strengthens a representation dynamic already present at lower magnitude in base and instruction-tuned LLMs. Our codebase is open-sourced \href{https://github.com/AI4Collaboration/Fluid-Reasoning-Representation}{here}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。