更大束宽未必更好,噪声评分会导致推理质量下降。
More Test-Time Compute Can Hurt: Overestimation Bias in LLM Beam Search
- 基于极值理论分析束搜索的过估计偏差,推导出最优束宽公式。
- 困惑度评分下最优束宽为1,而评分更准时可提升至4以上,增益达8.9%。
- 不同评分器导致束宽效果差异巨大,关键看信号噪声比。
更宽的束搜索本应提升大模型推理性能,但何时应停止加宽?以往研究关注推理效率,未分析更宽搜索是否反而损害输出质量。本文基于极值理论分析发现,对噪声评分结果进行束选择会引入系统性过估计偏差,且随候选池增大而加剧。我们推导出最大有效束宽$ ilde{k}$,其与评分器的信噪比$(Δ/σ)^2$呈指数关系:$Δ$为正确路径相对于错误路径的质量优势,$σ$为评分噪声。在三个70亿参数模型和十类任务上,使用MR-BEN数据集(5,975个问题)验证:困惑度评分因噪声高,最优束宽$ ilde{k} = 1$,任何宽度均无提升;而评分更准的PRM,$ ilde{k} \>= 4$,最高提升8.9个百分点。同一模型、算法,仅因评分器不同,最优束宽可相差悬殊。分析表明,评分器信噪比是决定束宽选择的核心因素,并提出实用诊断指标。
原文摘要 · Abstract (English)
Wider beam search should improve LLM reasoning, but when should you stop widening? Prior work on beam width selection has focused on inference efficiency \citep{qin2025dsbd, freitag2017beam}, without analyzing whether wider search can \emph{hurt} output quality. We present an analysis, grounded in Extreme Value Theory, that answers this question. Beam selection over noisy scorer outputs introduces a systematic overestimation bias that grows with the candidate pool size, and we derive a maximum useful beam width $\hat{k}$ beyond which search degrades performance. This critical width depends on the signal-to-noise ratio of the scorer: $\hat{k}$ grows exponentially with $(Δ/σ)^2$, where $Δ> 0$ is the quality advantage of correct paths over incorrect ones and $σ$ is the scorer noise. We validate this theory by comparing perplexity-guided and PRM-guided beam search across three 7B-parameter models and ten domains on MR-BEN (5,975 questions). Perplexity scoring, with its high noise, yields $\hat{k} = 1$: search provides no benefit at any width tested. PRM scoring, with lower noise, yields $\hat{k} \geq 4$, with gains of up to 8.9 percentage points. The same model, the same algorithm, but different scorers place $\hat{k}$ at opposite ends of the beam width range. Our analysis identifies the scorer's signal-to-noise ratio as the key quantity governing beam width selection, and we propose diagnostic indicators for choosing the beam width in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。