用熵值指导搜索分支，让大模型推理更高效准确
Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration
- 基于熵值判断不确定性，仅在关键位置扩展搜索树
- 多模型多数据集上优于多链采样，通过率更高
- 统一探索效率与不确定性估计，适合需要可靠推理的场景
大型语言模型在推理任务中表现强劲，但现有解码策略或盲目探索（随机采样），或冗余重复（独立多采样）。本文提出熵树（Entropy-Tree），一种基于树结构的解码方法，利用熵作为分支决策信号——仅在模型真正不确定的位置扩展搜索树。熵树在推理任务中表现出更优的准确率与校准性：在多个模型和数据集上，其 pass@k 优于 Multi-chain；预测熵的 AUROC 也优于多种传统指标。该方法将高效结构化探索与可靠不确定性估计统一于单一解码流程中。
原文摘要 · Abstract (English)
Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that exploits entropy as a signal for branching decisions--expanding the search tree only at positions where the model exhibits genuine uncertainty. Entropy-Tree shows superior accuracy and calibration in reasoning tasks: it achieves better pass@k than Multi-chain across multiple models and datasets, and its predictive entropy demonstrates better AUROC compared to several traditional metrics. Entropy-Tree unifies efficient structured exploration and reliable uncertainty estimation within a single decoding procedure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。