arXiv:2504.21801cs.CLcs.AI2025-04被引 281

用强化学习分解数学证明子目标,提升形式化推理能力

DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition

论文配图:DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
图 1 · 摘自论文原文
  • 通过递归证明管道生成初始训练数据,融合非形式与形式化推理
  • 在MiniF2F-test上达到88.9%通过率,解决PutnamBench中49题
  • 首次在AIME竞赛题中实现6道形式化求解,逼近人类水平

我们提出DeepSeek-Prover-V2,一个面向Lean 4的形式化定理证明开源大模型,其初始化数据由DeepSeek-V3驱动的递归证明管道收集。冷启动训练首先让DeepSeek-V3将复杂问题分解为一系列子目标,再将已解决子目标的证明合成链式思维过程,结合DeepSeek-V3的逐步推理,构建强化学习的初始训练数据。该方法实现了非形式与形式化数学推理的统一建模。最终模型DeepSeek-Prover-V2-671B在神经定理证明任务中达到领先性能:在MiniF2F-test上通过率达88.9%,在PutnamBench中解决49道题(共658题)。此外,我们引入新基准ProverBench,包含325个形式化问题,其中15道来自近年AIME竞赛(2024–2025年)。模型在这些题中成功求解6道,而DeepSeek-V3通过多数投票可解8道,表明大模型在形式与非形式数学推理间的差距显著缩小。

原文摘要 · Abstract (English)

We introduce DeepSeek-Prover-V2, an open-source large language model designed for formal theorem proving in Lean 4, with initialization data collected through a recursive theorem proving pipeline powered by DeepSeek-V3. The cold-start training procedure begins by prompting DeepSeek-V3 to decompose complex problems into a series of subgoals. The proofs of resolved subgoals are synthesized into a chain-of-thought process, combined with DeepSeek-V3's step-by-step reasoning, to create an initial cold start for reinforcement learning. This process enables us to integrate both informal and formal mathematical reasoning into a unified model. The resulting model, DeepSeek-Prover-V2-671B, achieves state-of-the-art performance in neural theorem proving, reaching 88.9% pass ratio on the MiniF2F-test and solving 49 out of 658 problems from PutnamBench. In addition to standard benchmarks, we introduce ProverBench, a collection of 325 formalized problems, to enrich our evaluation, including 15 selected problems from the recent AIME competitions (years 24-25). Further evaluation on these 15 AIME problems shows that the model successfully solves 6 of them. In comparison, DeepSeek-V3 solves 8 of these problems using majority voting, highlighting that the gap between formal and informal mathematical reasoning in large language models is substantially narrowing.

形式化证明强化学习数学推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。