用信息量信号在推理时剪枝思维链，大幅降耗不丢精度。
Inference-Time Chain-of-Thought Pruning with Latent Informativeness Signals
- 融合KL散度、置信度和熵构造评分函数，动态评估路径质量。
- 相比BoN，峰值内存降60%，总生成token减90%，精度几乎不变。
- 适合资源受限场景，尤其对小模型性能提升明显。
大型语言模型在测试时生成多个候选解可提升推理准确率，但标准方法如Best-of-N（BoN）需完全生成所有分支，计算成本高。Self-Truncation Best-of-N（ST-BoN）通过早期截断无希望路径缓解此问题，但依赖一致性启发式，无法直接评估分支质量。本文提出KL-Adjust Pruned Path Algorithm（KAPPA），在推理时结合Kullback-Leibler散度、置信度与熵构建合理评分函数，指导渐进式剪枝。该方法在探索阶段促进多样性，有选择地剔除低分路径，在保持准确率的同时显著降低内存与令牌使用量。在GSM8K与MATH500数据集上，基于DeepSeek-R1-Distill-Qwen-1.5B与Qwen2.5-7B-Instruct的实验表明，KAPPA使小模型性能更稳定，并实现相对于BoN约60%的峰值内存减少与约90%的总令牌生成减少，精度损失极小。
原文摘要 · Abstract (English)
Large language models (LLMs) improve reasoning accuracy when generating multiple candidate solutions at test time, but standard methods like Best-of-N (BoN) incur high computational cost by fully generating all branches. Self-Truncation Best-of-N (ST-BoN) mitigates this by truncating unpromising paths early, but its reliance on consistency-based heuristics is a limitation as it does not directly evaluate branch quality. We present KL-Adjusted Pruned Path Algorithm (KAPPA), an inference-time method that combines Kullback-Leibler divergence, confidence, and entropy into a principled scoring function to guide progressive pruning. By promoting diversity during exploration and selectively eliminating low-scoring branches, KAPPA maintains accuracy while substantially reducing memory and token usage. Experiments on GSM8K and MATH500 with DeepSeek-R1-Distill-Qwen-1.5B and Qwen2.5-7B-Instruct demonstrate that KAPPA stabilizes performance in smaller models and achieves up to ~60% reduction in peak memory and ~90% reduction in total token generation relative to BoN, with minimal impact on accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。