arXiv:2504.10481cs.CL2025-04被引 48

高效验证推理模型答案正确性,准确率超95%。

xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

  • 设计新验证器xVerify,精准判断复杂推理答案是否等价。
  • 在多类题目上,所有版本F1和准确率均超95%。
  • 适合用于训练和评估大模型推理能力,尤其适配长推理场景。

随着OpenAI o1模型的发布,采用慢思考策略的推理模型日益普遍。其输出常包含复杂推理过程、中间步骤与自我反思,现有评估方法与奖励模型难以应对。尤其在判断答案等价性和从长文本中提取最终答案方面表现不足。为此,我们提出xVerify,一种高效的答案验证器,用于评估推理模型。xVerify具备强等价判断能力,可跨多种题型准确比对模型输出与参考答案。为训练与评估xVerify,我们构建了VAR数据集,该数据集由多个LLM在不同数据集上生成的问答对组成,涵盖多种推理模型及专为推理评估设计的挑战性评测集,并采用多轮标注保障标签质量。基于VAR,我们训练了不同规模的xVerify模型。在测试集与泛化集上的实验结果表明,所有xVerify变体的F1分数和准确率均超过95%。值得注意的是,最小模型xVerify-0.5B-I的表现优于除GPT-4o外的所有评估方法,而xVerify-3B-Ib在整体性能上超越GPT-4o。此外,以xVerify作为奖励模型进行强化学习实验,使Qwen2.5-7B相比直接生成提升18.4%,优于使用Math Verify作为奖励的提升效果。这些结果证明了xVerify的有效性与泛化能力。所有xVerify资源已在GitHub公开。

原文摘要 · Abstract (English)

With the release of OpenAI's o1 model, reasoning models that adopt slow-thinking strategies have become increasingly common. Their outputs often contain complex reasoning, intermediate steps, and self-reflection, making existing evaluation methods and reward models inadequate. In particular, they struggle to judge answer equivalence and to reliably extract final answers from long, complex responses. To address this challenge, we propose xVerify, an efficient answer verifier for evaluating reasoning models. xVerify shows strong equivalence judgment capabilities, enabling accurate comparison between model outputs and reference answers across diverse question types. To train and evaluate xVerify, we construct the VAR dataset, which consists of question-answer pairs generated by multiple LLMs across various datasets. The dataset incorporates multiple reasoning models and challenging evaluation sets specifically designed for reasoning assessment, with a multi-round annotation process to ensure label quality. Based on VAR, we train xVerify models at different scales. Experimental results on both test and generalization sets show that all xVerify variants achieve over 95% F1 score and accuracy. Notably, the smallest model, xVerify-0.5B-I, outperforms all evaluation methods except GPT-4o, while xVerify-3B-Ib surpasses GPT-4o in overall performance. In addition, reinforcement learning experiments using xVerify as the reward model yield an 18.4% improvement for Qwen2.5-7B compared with direct generation, exceeding the gains achieved with Math Verify as the reward. These results demonstrate the effectiveness and generalizability of xVerify. All xVerify resources are available on \href{https://github.com/IAAR-Shanghai/xVerify}{GitHub}.

模型评估推理验证大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。