通过温度采样扩展推理能力,让大模型解题更准。
On the Role of Temperature Sampling in Test-Time Scaling
- 用不同温度生成多条推理路径,扩大模型解题范围。
- 在多个基准上比单一温度提升7.3分,突破原有性能瓶颈。
- 无需强化学习训练,就能逼近训练后模型效果。
大型语言模型在推理时可通过测试时扩展(TTS)提升推理能力,即生成多条推理路径并选择最优者。已有研究显示增加采样数量K可稳步提升准确率。本文发现该趋势并非无限延续:当K过大时,进一步扩展不再带来收益,部分难题仍无法解决。有趣的是,不同采样温度能解决不同问题子集,表明单温度扩展仅挖掘了模型潜力的一小部分。为此,我们提出沿温度维度扩展的方法,显著拓宽了模型的推理边界。在Qwen3(0.6B、1.7B、4B、8B)和五个代表性推理基准(AIME 2024/2025、MATH500、LiveCodeBench、Hi-ToM)上的平均结果表明,温度扩展相较单温度TTS额外提升7.3分。该方法还使基础模型达到与强化学习训练模型相当的性能,无需额外后训练。我们进一步分析现象本质,并设计多温度投票机制以降低计算开销。总体而言,本研究揭示了TTS的潜力远超以往认知,温度扩展是一种简单有效的释放基模型潜能的方式。
原文摘要 · Abstract (English)
Large language models (LLMs) can improve reasoning at inference time through test-time scaling (TTS), where multiple reasoning traces are generated and the best one is selected. Prior work shows that increasing the number of samples K steadily improves accuracy. In this paper, we demonstrate that this trend does not hold indefinitely: at large K, further scaling yields no gains, and certain hard questions remain unsolved regardless of the number of traces. Interestingly, we find that different sampling temperatures solve different subsets of problems, implying that single-temperature scaling explores only part of a model's potential. We therefore propose scaling along the temperature dimension, which enlarges the reasoning boundary of LLMs. Averaged over Qwen3 (0.6B, 1.7B, 4B, 8B) and five representative reasoning benchmarks (AIME 2024/2025, MATH500, LiveCodeBench, Hi-ToM), temperature scaling yields an additional 7.3 points over single-temperature TTS. Temperature scaling also enables base models to reach performance comparable to reinforcement learning (RL)-trained counterparts, without additional post-training. We further provide a comprehensive analysis of this phenomenon and design a multi-temperature voting method that reduces the overhead of temperature scaling. Overall, our findings suggest that TTS is more powerful than previously thought, and that temperature scaling offers a simple and effective way to unlock the latent potential of base models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。