用解题成功率提升评估大模型推理前缀，更精准指导求解路径。
From Correctness to Utility: Gain-Based Prefix Evaluation for LLM Reasoning
- 以解题成功率提升定义前缀收益，替代局部正确性评估。
- 在数学推理任务中，前缀收益评分显著提升搜索效率与成功率。
- 适合需要高效筛选推理路径的研究者与工程落地场景。
推理前缀决定大模型解题的未来走向，但现有过程奖励模型多依赖局部步骤正确性评估。我们认为正确性仅为间接指标，真正关键的是前缀能否提升最终解题成功率。为此提出前缀收益（prefix gain）——即条件化轻量学生模型组时带来的求解率提升，并基于此构建前缀效用模型（PUM），采用简单成对排序目标进行训练。PUM学习以结果为导向的前缀效用，可对完整轨迹和部分推理前缀进行评分。在Best-of-$N$选择、束搜索及强化学习等场景中，尤其当候选池大、搜索预算高或规则奖励稀疏时，PUM提供强前缀级监督信号。所有数据、模型与代码已公开于https://zhiqix.github.io/pum-project-page。
原文摘要 · Abstract (English)
Reasoning prefixes shape the future trajectory of LLM problem solving, yet existing process reward models usually evaluate them through local step correctness. We argue that correctness is a useful but indirect proxy for the effect we ultimately care about: whether a prefix increases the probability of successful completion. We define this effect as prefix gain, the solve-rate improvement induced by conditioning lightweight student model group on a prefix, and use it to train a Prefix Utility Model (PUM) with a simple pairwise ranking objective. PUM learns outcome-grounded prefix utility and can score both complete trajectories and partial reasoning prefixes. Across Best-of-$N$ selection, beam search, and reinforcement learning on mathematical reasoning, PUM provides a strong prefix-level supervision signal, especially when candidate pools are large, search budgets increase, or rule-based rewards are sparse. We release all data, models, and code at https://zhiqix.github.io/pum-project-page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。