选最短答案能大幅提升大模型推理效果,还更省算力。
The Virtues of Brevity: Avoid Overthinking in Parallel Test-Time Reasoning
- 用最短答案作为最优解,避免复杂评分。
- 在两个基准上表现媲美自洽性方法,但耗时减少50%以上。
- 适合追求高效推理的开发者和实际部署场景。
推理模型显著提升了大语言模型在数学与编程等复杂任务上的能力。以往研究证明,通过并行测试时计算(采样多个解并择优)可进一步提升性能,但此类方法常需复杂评分机制,增加计算开销。本文发现一种简单且反直觉的启发式策略:选择最短解极为有效。我们提出,模型在两种不同行为模式间切换:简洁自信的常规模式与冗长犹豫的过度思考模式,并实证发现存在一个临界点,超过该点后过度思考开始主导。选择最短答案相当于优先采样常规模式。实验表明,该方法在两个挑战性基准上性能媲美自洽性方法,同时大幅降低计算开销。最短答案启发式在输出无明确相等定义的任务中也适用,带来帕累托改进。
原文摘要 · Abstract (English)
Reasoning models represent a significant advance in LLM capabilities, particularly for complex reasoning tasks such as mathematics and coding. Previous studies confirm that parallel test-time compute-sampling multiple solutions and selecting the best one-can further enhance the predictive performance of LLMs. However, strategies in this area often require complex scoring, thus increasing computational cost and complexity. In this work, we demonstrate that the simple and counterintuitive heuristic of selecting the shortest solution is highly effective. We posit that the observed effectiveness stems from models operating in two distinct regimes: a concise, confident conventional regime and a verbose overthinking regime characterized by uncertainty, and we show evidence of a critical point where the overthinking regime begins to be significant. By selecting the shortest answer, the heuristic preferentially samples from the conventional regime. We confirm that this approach is competitive with more complex methods such as self-consistency across two challenging benchmarks while significantly reducing computational overhead. The shortest-answer heuristic provides a Pareto improvement over self-consistency and applies even to tasks where output equality is not well defined.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。