用熵变化判断大模型何时该推理,实现高效自适应生成。
When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

- 通过早期解码熵的变化识别推理状态
- 实现41%-55%的显著降耗且提升准确率
- 无需训练,适合各类大模型快速部署
链式思维(CoT)已成为提升大模型能力的通用策略,但其适用时机仍存根本疑问:何时显式推理真正有益?实证发现,CoT在事实性与开放性任务中常带来边际甚至负面收益,同时显著增加令牌消耗。本文揭示,模型推理并非任务或模型的静态属性,而是一种在生成过程中动态涌现的解码状态。系统分析表明,早期熵动态可可靠指示该状态:受益于CoT的任务呈现稳定的熵下降趋势,而其他任务则表现为不稳或上升模式。这一行为可被解释为从高熵探索态向低熵结构化推理态的相变过程。基于此,我们提出轻量级、免训练的路由框架EDRM(熵动力学推理流形),利用早期解码熵来自适应选择推理策略。EDRM将熵轨迹嵌入紧凑可解释的流形表示,支持零样本部署与实例级精细化调整。在15个基准和4种不同规模与架构的大模型上,EDRM持续优于静态基线。在数据集层面,仅需50个校准样本即可实现41–55%的令牌减少并提升准确率;在实例层面,准确率最高提升4.7%,同时保持27–45%的令牌节省。结果表明,推理应按需调用而非默认开启,并验证了熵驱动解码控制在高效自适应大模型推理中的有效性。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning has become the default strategy for enhancing LLM capabilities, yet its application raises a fundamental question: when is explicit reasoning actually beneficial? Empirical evidence reveals a striking paradox: CoT often provides marginal or even negative gains on factual and open-ended tasks while multiplying token consumption. In this work, we show that LLM reasoning is not a static property of tasks or models, but a \emph{dynamic decoding state} that emerges during generation. Through systematic analysis, we find early-stage entropy dynamics provide a reliable signal of this state: tasks benefiting from CoT exhibit consistent entropy reduction, while others display unstable or increasing patterns. This behavior can be interpreted as a phase-transition-like shift from a high-entropy exploratory regime to a low-entropy structured reasoning regime. Based on these insights, we propose \textbf{EDRM} (Entropy Dynamics-based Reasoning Manifold), a lightweight and training-free routing framework that leverages early decoding entropy to adaptively select inference strategies. EDRM embeds entropy trajectories into a compact and interpretable manifold representation, enabling both zero-shot deployment and fine-grained instance-level adaptation. Across 15 benchmarks and 4 LLMs of varying scales and architectures, EDRM consistently outperforms static baselines. At the dataset level, EDRM achieves \textbf{41--55\%} token reduction while improving accuracy with as few as 50 calibration samples. At the instance level, it further improves accuracy by up to \textbf{4.7\%} while maintaining \textbf{27--45\%} token savings. These results suggest that reasoning should be invoked selectively rather than by default, and demonstrate the effectiveness of entropy-driven decoding control for efficient and adaptive LLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。