arXiv:2505.13903cs.CLcs.AI2025-05KDD被引 16

构建数学题质量验证框架,提升大模型训练数据可靠性

Let's Verify Math Questions Step by Step

  • 将数学题拆解为原子假设与结论,通过语义一致性检查验证正确性
  • 在2147道人工验证题上实现95%以上准确率,比基线提升25个百分点F1
  • 适合需高质量数学数据的模型训练与数据清洗场景

大型语言模型在数学推理方面取得显著进展,但现有工作多关注生成正确推理路径与答案,忽视题目本身正确性。本文提出ValiMath基准,包含2147道来自NuminaMath数据集、经人工验证的数学题,覆盖算术、代数、几何等多领域,每题标注逻辑结构、领域分布与正确性,支持细粒度评估。基于此,我们设计MathQ-Verify管道,对数学题进行细粒度解析,通过一致性检查验证其语义合理性。实验表明,该方法在多个基准上达到领先性能,相比直接验证基线,F1分数最高提升25个百分点。该方案可有效降低数据噪声,避免无效计算,为数学数据集清洗提供可扩展、高精度解决方案。代码与数据已开源。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have recently achieved remarkable progress in mathematical reasoning. To enable such capabilities, many existing works distill strong reasoning models into long chains of thought or design algorithms to construct high-quality math question-answer (QA) data for training. However, these efforts primarily focus on generating correct reasoning paths and answers, while largely overlooking the correctness of the questions themselves. In this work, we present ValiMath, a benchmark consisting of 2147 human-verified mathematical questions covering a wide range of domains such as arithmetic, algebra, and geometry, which are synthesized and curated from the NuminaMath dataset. Each question is annotated with its logical structure, domain coverage, and question correctness, enabling fine-grained evaluation of question quality. ValiMath serves as a high-quality gold-standard test set for validating mathematical questions in LLM training corpora. Building upon this benchmark, we further propose MathQ-Verify, a pipeline that performs fine-grained parsing of mathematical questions into atomic assumptions and conclusions, and evaluates their semantic soundness through consistency checks. This pipeline achieves high precision in detecting flawed questions and provides a reliable foundation for cleaning noisy mathematical datasets. Experiments show that MathQ-Verify achieves state-of-the-art performance across multiple benchmarks, improving the F1 score by up to 25 percentage points over the direct verification baseline. MathQ-Verify offers a scalable and accurate solution for curating reliable mathematical datasets, reducing label noise and avoiding unnecessary computation on invalid questions. Our code and data are available at the repository https://github.com/OpenDCAI/MathQ-Verify.

数学推理数据验证大模型训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。