arXiv:2608.18931cs.CLcs.AI2026-08

测试时扩展的瓶颈在于选择,而非生成更多候选。

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

论文配图:Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck
图 1 · 摘自论文原文
  • 拆解推理计算为探索与利用,分析各方法效果。
  • 探索能力随算力提升持续增强,但利用效率极低。
  • 适合关注推理优化、模型选择机制的研究者。

测试时扩展(TTS)通过增加推理计算来提升语言模型输出质量,如生成多个候选、搜索部分序列或迭代精炼草稿。这些技术在数学和代码任务中表现优异,但主要在验证简单的任务上开发和测试。本文首次在五个开放生成基准(医学、法律、金融、通用对话、创意写作)上,以统一框架进行算力归一化比较五种TTS方法,将算力预算分解为探索与利用两部分。结果显示:探索能力随算力提升稳步增强;但利用环节失败——最先进的生成器中,奖励模型与真实质量的相关性仅约ρ_v ≈ 0.12,导致选择近乎随机,无论预算多少。树搜索因多样性崩溃加剧此问题。精炼仅在一项任务有效,其余“增益”存在混淆。唯有跨候选合成(Fusion)持续优于单样本基线,但仍仅恢复约40%可用质量。真正的瓶颈在于从候选池中选择,而非候选池本身。

原文摘要 · Abstract (English)

Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only $ρ_v \approx 0.12$ with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.

测试时扩展推理优化模型选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。