arXiv:2506.02867cs.AIcs.CL2025-06NeurIPS被引 61

发现大模型推理时的思维标记是信息峰值,能提升解题能力。

Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning

  • 用互信息追踪推理过程,发现特定生成步骤出现信息突增。
  • 信息峰值对应'Hmm'等思考标记词,其出现显著降低错误率。
  • 利用这些思考标记可简单提升模型推理性能,适合优化大模型。

大型推理模型(LRMs)在复杂问题求解中表现优异,但其内部推理机制仍不清晰。本文从信息论视角研究LRM的推理轨迹,通过追踪中间表示与正确答案间的互信息(MI)演变,发现推理过程中存在显著的MI峰值现象:在某些生成步骤,互信息突然大幅上升。理论分析表明,随着MI增加,模型预测错误概率下降。这些峰值常出现在表达反思或转折的词语(如'Hmm'、'Wait'、'Therefore')处,我们称之为思考标记(thinking tokens)。实验显示,这些标记对推理性能至关重要,而其他词汇影响较小。基于此,我们提出两种简洁有效的改进方法,通过巧妙利用思考标记提升推理表现。本工作为理解大模型推理机制提供新视角,并给出实用优化路径。代码已公开于https://github.com/ChnQ/MI-Peaks。

原文摘要 · Abstract (English)

Large reasoning models (LRMs) have demonstrated impressive capabilities in complex problem-solving, yet their internal reasoning mechanisms remain poorly understood. In this paper, we investigate the reasoning trajectories of LRMs from an information-theoretic perspective. By tracking how mutual information (MI) between intermediate representations and the correct answer evolves during LRM reasoning, we observe an interesting MI peaks phenomenon: the MI at specific generative steps exhibits a sudden and significant increase during LRM's reasoning process. We theoretically analyze such phenomenon and show that as MI increases, the probability of model's prediction error decreases. Furthermore, these MI peaks often correspond to tokens expressing reflection or transition, such as ``Hmm'', ``Wait'' and ``Therefore,'' which we term as the thinking tokens. We then demonstrate that these thinking tokens are crucial for LRM's reasoning performance, while other tokens has minimal impacts. Building on these analyses, we propose two simple yet effective methods to improve LRM's reasoning performance, by delicately leveraging these thinking tokens. Overall, our work provides novel insights into the reasoning mechanisms of LRMs and offers practical ways to improve their reasoning capabilities. The code is available at https://github.com/ChnQ/MI-Peaks.

大模型推理信息论思维标记注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。