用大模型长推理选最优解，高效提升数学推理能力。
GenSelect: A Generative Approach to Best-of-N
- 让大模型通过长篇推理从N个答案中选出最佳解。
- 在数学推理任务上，性能超越传统打分与配对比较方法。
- 适合需要高质量推理输出的场景，如复杂问题求解。
具有并行采样的生成式奖励模型已实现推理任务的测试时扩展。现有方法采用单个解的点对点评分或成对比较，但点对点方法未能充分利用大模型的比较能力，而成对方法在更大采样预算下效率低下。我们提出GenSelect，让大模型通过长推理从N个候选解中选择最优解，既发挥其比较优势，又在并行采样规模上高效扩展。在数学推理任务中，我们发现像QwQ和DeepSeek-R1-0528这样的推理模型在简单提示下即可显著优于现有打分方法。
原文摘要 · Abstract (English)
Generative reward models with parallel sampling have enabled effective test-time scaling for reasoning tasks. Current approaches employ pointwise scoring of individual solutions or pairwise comparisons. However, pointwise methods underutilize LLMs' comparative abilities, while pairwise methods scale inefficiently with larger sampling budgets. We introduce GenSelect, where the LLM uses long reasoning to select the best solution among N candidates. This leverages LLMs' comparative strengths while scaling efficiently across parallel sampling budgets. For math reasoning, we demonstrate that reasoning models, such as QwQ and DeepSeek-R1-0528, excel at GenSelect, outperforming existing scoring approaches with simple prompting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。