verifier引导搜索在数学推理中随样本量增大反而失效，因验证器会误删正确路径。
Scaling Flaws of Verifier-Guided Search in Mathematical Reasoning
- 用验证器筛选推理路径，优先保留有效解法。
- 样本量增大时性能反不如重复采样，尤其在难题上更差。
- 适合研究推理机制缺陷或改进验证策略的读者。
大语言模型在多步推理任务中表现不佳，而推理时扩展（inference-time scaling）成为提升性能的可行策略。验证器引导搜索在样本受限时优于重复采样，能筛选并优先处理有效推理路径。然而我们发现，该方法存在普遍性的缩放缺陷：随着样本数量增加，其优势逐渐减弱，最终低于重复采样。这一现象在多个模型（Mistral 7B、DeepSeekMath 7B）、基准数据集（GSM8K、MATH）和验证器类型（结果值模型、过程奖励模型）中均出现。分析表明，问题根源在于验证器失败——不完美的验证器会错误排序候选路径，导致所有有效路径被误删。此类问题在复杂及分布外任务中尤为严重，削弱了搜索的有效性。为缓解验证器依赖，我们探索了两种简化方法，初步结果显示验证器引导搜索存在根本性局限，提示未来研究方向。
原文摘要 · Abstract (English)
Large language models (LLMs) struggle with multi-step reasoning, where inference-time scaling has emerged as a promising strategy for performance improvement. Verifier-guided search outperforms repeated sampling when sample size is limited by selecting and prioritizing valid reasoning paths. However, we identify a critical limitation: scaling flaws, prevalent across different models (Mistral 7B and DeepSeekMath 7B), benchmarks (GSM8K and MATH), and verifiers (outcome value models and process reward models). As sample size increases, verifier-guided search exhibits diminishing advantages and eventually underperforms repeated sampling. Our analysis attributes this to verifier failures, where imperfect verifiers misrank candidates and erroneously prune all valid paths. These issues are further exacerbated in challenging and out-of-distribution problems, restricting search effectiveness. To mitigate verifier failures, we explore reducing reliance on verifiers and conduct preliminary investigations using two simple methods. Our findings reveal fundamental limitations in verifier-guided search and suggest future directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。