arXiv:2606.10799cs.AI2026-06

用逐步验证提升大模型对数学证明的严谨审查能力。

Evaluating Research-Level Math Proofs via Strict Step-Level Verification

  • 逐步追踪每一步推理,严格限制定理使用来源。
  • 在研究级证明测试中,错误定位准确率显著优于全局评估。
  • 暴露专家基准中的隐含歧义,适合数学推理与自动审稿研究者。

大型语言模型在严谨验证复杂数学证明时表现不佳。传统全局评估易受‘上下文污染’影响,看似合理的陈述可能掩盖细微逻辑漏洞,导致幻觉或过度怀疑。为此,本文转向严格的逐步验证:框架为每个推导步骤维护详细上下文,并严格约束定理应用来源。在从FirstProof挑战中精选的对抗性诊断数据集上评估,系统性消融实验表明,这些推导约束不可或缺,无约束的全局提示始终无法定位细微逻辑错误。相较于全局评估,本方法从根本上改变了失败模式。错误分析显示,剩余拒绝主要源于‘吹毛求疵的过度严谨’,来自未明确的领域惯例,实质上揭示了专家基准本身的隐含模糊性。研究结果表明,引导代理以谨慎、类人类数学家的方式组织验证笔记,可显著提升其区分严谨证明与有缺陷证明的能力,具备强化代理在前沿数学概念上的推理潜力,并为未来自动化证明评审系统奠定理论基础。代码与提示已开源于GitHub。

原文摘要 · Abstract (English)

Large Language Models (LLMs) struggle to rigorously verify complex mathematical proofs. Standard global evaluation approaches suffer from "context poisoning," in which superficially plausible statements mask subtle logical flaws, leading to hallucination or over-skepticism. To address this, we shift from global evaluation to strict step-level verification: our framework maintains detailed context for each deduction step and strictly constrains the sources of applied theorems. We evaluate on a carefully curated adversarial diagnostic suite of research-level proofs drawn from the FirstProof challenge. A systematic ablation study demonstrates that these deductive constraints are indispensable, as unconstrained global prompting consistently fails to localize subtle logical errors. Beyond outperforming global evaluation, our approach fundamentally alters the failure taxonomy. Error analysis reveals that, rather than exhibiting severe logical hallucinations, remaining rejections are primarily instances of "pedantic hyper-rigor" stemming from unstated domain conventions, effectively exposing implicit ambiguities within the expert benchmark itself. Our findings suggest that prompting agents to organize their verification notes in a cautious, human-mathematician-like manner can substantially improve their ability to distinguish rigorous proofs from flawed ones, with the potential to strengthen agentic reasoning on frontier mathematical concepts that the base model does not already know well, and to lay a theoretical foundation for future automated proof-review systems. Code and prompts are available at GitHub.

数学证明推理验证大模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。