采样越多越差?研究发现关键答案在几十次内就已确定。
When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling
- 通过分析选择机制,发现多数样本中正确答案早已出现
- 投票结果在数十次采样后即稳定,后续无增益
- 适合关注推理效率与资源优化的研究者
语言模型通过多次采样(测试时扩展)来回答难题,覆盖率随采样次数上升,但最终需从中选出一个答案。由于无法识别哪次采样正确,选择过程受限于‘可辨识性’。本文指出,正确答案在几十次采样内已足够被覆盖,之后的额外采样仅增强错误答案的自信,反而降低质量。核心瓶颈是识别正确答案而非生成它。文中提出‘有效采样数’这一指标,能从任意采样运行中直接揭示最优采样量,表明继续采样既浪费算力又无益,甚至有害。关键结论:选答案的投票在几十次内已定局,而基准得分更早达到峰值。
原文摘要 · Abstract (English)
People overthink; language models over-sample, and the extra effort can talk both into a worse answer. Reasoning systems answer a hard question by sampling it many times (test-time scaling), and the more they draw, the more often a correct answer turns up somewhere, so coverage, the fraction of problems with at least one correct try, climbs and appears to be progress. But a deployed system must return one answer, and choosing it, not knowing which try is right, is selection; selection is capped, and past a point extra samples only make the model surer of a confident mistake, even as every draw adds cost. The gap between climbing coverage and stalled selection, the identifiability gap, is the answer a model can produce but not pick. So the real question is not whether to sample but how far, and the answer is: not far. For picking an answer, the vote has already settled within a few dozen draws, the modal ceiling; for scoring a benchmark, sooner still, the correlation ceiling. Beyond that, extra draws cost compute and add nothing, and can even make the answer worse. This paper turns the cutoff into a single number, the effective number of samples, that any sampling run already reveals. The bottleneck is recognizing a right answer, not generating one.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。