用物理力学分析大模型推理路径，发现正确推理有更低能量值。
Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs
- 将推理链映射为哈密顿系统，用动能与势能平衡衡量信息获取与问题相关性。
- 有效推理的哈密顿能量值更低，体现信息收集与目标回答的最优权衡。
- 适合研究模型推理机制的学者，提供新视角和可视化诊断工具。
本文提出一种基于哈密顿力学的新方法，用于分析语言模型中的多跳推理。我们将问答数据集中推理链在嵌入空间中的轨迹映射为哈密顿系统，定义一个函数，平衡推理进展（动能）与问题相关性（势能）。分析表明，有效推理具有更低的哈密顿能量值，反映了信息收集与目标回答之间的最优权衡。尽管该框架提供了复杂的可视化与量化手段，但其宣称可“引导”或“改进”推理算法仍需更严格的实证验证，因物理系统与推理过程间的联系仍主要为隐喻。然而，我们的分析揭示了区分有效与无效推理的稳定几何模式，表明这一受物理启发的方法为理解大模型推理过程提供了有前景的诊断工具与新视角。
原文摘要 · Abstract (English)
This paper proposes a novel approach to analyzing multi-hop reasoning in language models through Hamiltonian mechanics. We map reasoning chains in embedding spaces to Hamiltonian systems, defining a function that balances reasoning progression (kinetic energy) against question relevance (potential energy). Analyzing reasoning chains from a question-answering dataset reveals that valid reasoning shows lower Hamiltonian energy values, representing an optimal trade-off between information gathering and targeted answering. While our framework offers complex visualization and quantification methods, the claimed ability to "steer" or "improve" reasoning algorithms requires more rigorous empirical validation, as the connection between physical systems and reasoning remains largely metaphorical. Nevertheless, our analysis reveals consistent geometric patterns distinguishing valid reasoning, suggesting this physics-inspired approach offers promising diagnostic tools and new perspectives on reasoning processes in large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。