发现思维链推理的熵变化有探索到收敛的两阶段规律,可用来高效控制推理过程。
Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning

- 将思维链推理的熵变化建模为序贯变点检测问题,用CUSUM算法实时识别收敛阶段。
- 在不增加训练成本前提下,实现63.06%准确率同时减少11.1%计算量,优于现有方法。
- 适合需要高效推理的场景,如资源受限部署或大规模生成任务。
本文研究思维链(CoT)推理中的熵动态,发现其具有稳定的两阶段结构:探索期的不确定性逐渐过渡到收敛期的确定性。在收敛阶段,模型表现出两个关键特性:1)高可靠性——答案趋于准确且稳定;2)高冗余性——模型在得出正确答案后仍生成大量无意义的后续词元。基于此,提出两种高效推理策略:1)早退机制利用可靠性和冗余性,在收益下降时安全终止推理;2)测试时扩展通过收敛信号优先选择已收敛轨迹。为实现这一洞察,首次将经典变点检测方法应用于监控CoT推理过程。采用统计最优的累积和(CUSUM)算法,构建无需训练的实时推理控制框架。实验表明,该方法在早退任务中建立更优的帕累托前沿:在63.06%准确率下实现11.1%的词元减少,相比DEER和Dynasor分别提升3.28%和4.36%的准确率。在测试时扩展任务中,基于CUSUM加权投票始终优于自一致性方法。
原文摘要 · Abstract (English)
This paper investigates the entropy dynamics of Chain-of-Thought (CoT) and uncovers a consistent two-phase structure: an Uncertainty Region of exploration transitioning sharply to a Confidence Region of convergence. We demonstrate that the Confidence Region possesses two critical properties: 1) High Reliability -- answers in the confidence region become highly accurate and stable, and 2) High Redundancy -- models generate unnecessary tokens long after reaching the correct answer. These properties unlock more efficient and reliable inference strategies: 1) Early Exit leverages reliability and redundancy to terminate computation safely when returns diminish, and 2)Test-Time Scaling uses the Confidence Region signal to prioritize converged trajectories. To operationalize these insights, we formulate Confidence Region detection as a sequential change-point detection problem, being the first to apply classical change-point methods to monitor CoT reasoning. Using the Cumulative Sum (CUSUM) algorithm, a statistically optimal change-point detector, we develop a training-free framework for real-time inference control. Experiments show our approach establishes a superior Pareto-frontier for early exit. CUSUM achieves 63.06% accuracy with 11.1% token reduction, outperforming DEER and Dynasor by 3.28% and 4.36% in accuracy respectively. For test-time scaling, CUSUM-weighted voting consistently outperforms self-consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。