自适应停止推理生成，节省30%以上token与延迟。
Logit-Entropy Adaptive Stopping Heuristic for Efficient Chain-of-Thought Reasoning
- 根据词元熵斜率和顶级对数几率差判断是否停止生成。
- 在GSM8K等数据集上减少30%-35%的生成token，延迟降27%。
- 无需训练即可使用，适合追求效率的推理部署场景。
链式思维（CoT）提示是激发大模型复杂推理能力的关键技术。然而，生成固定长度的完整推理过程计算开销大，导致令牌消耗和延迟增加。我们提出LEASH：一种无需训练的解码启发式算法，可自适应终止推理生成。LEASH监测两个内在信号：词元级熵的斜率和顶级对数几率差的改善。当两者均趋于平稳时，表示模型已进入稳定推理状态，生成即终止。在四个指令微调模型上，于GSM8K和AQuA-RAT基准测试中，LEASH平均减少30%–35%的令牌生成量，延迟降低27%，但相比标准CoT有10个百分点的准确率下降。该方法具有模型无关性，无需额外训练或监督，是一种简单高效的CoT解码替代方案。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting is a key technique for enabling complex reasoning in large language models. However, generating full, fixed-length rationales is computationally wasteful, inflating both token usage and latency. We introduce LEASH: Logit-Entropy Adaptive Stopping Heuristic, a training-free decoding algorithm that adaptively halts rationale generation. LEASH monitors two intrinsic signals: the slope of token-level entropy and the improvement in the top-logit margin. It terminates the generation once both signals plateau, indicating the model has reached a stable reasoning state. Across four instruction-tuned models on the GSM8K and AQuA-RAT benchmarks, LEASH reduces average token generation by 30--35% and latency by 27%, while incurring a 10 p.p. accuracy drop relative to CoT. LEASH is model-agnostic and requires no additional training or supervision, offering a simple and efficient alternative to CoT decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。