arXiv:2501.18310cs.LGcs.AI2025-01ICML被引 13

通过细粒度分析提升大模型证明效率,实现更少查询更高成功率。

ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis

  • 基于生成证明的结构分析,分层级调用自动化工具增强推理。
  • 在miniF2F上仅用2100次查询达66.0%通过率,远超前人56.1%。
  • 可插拔适配任意搜索算法,适合追求高效数学证明的开发者。

深度学习模型与传统自动化工具(如证明助手内置策略和现成自动定理证明器)的协同,在构建稳健高效的神经定理证明器(NTPs)中起关键作用。然而,现有基于大语言模型(LLM)的证明合成方法仅在模型显式调用时或单一粒度下使用自动化工具,未能充分挖掘其潜力。为此,我们提出ProofAug,一种通过细粒度分析模型生成的证明方案,将自动化方法以多粒度嵌入LLM。ProofAug可作为通用模块,无缝集成至任意树搜索算法,由此构建高效的递归证明(ERP)模块以进一步提升性能。在使用DeepSeek-Math-7B-base模型与Isabelle证明助手的miniF2F基准上验证了该方法的优越性。通过混合提示策略,经数据集清洗后累计通过率达66.0%(原始版本为61.9%),每题仅需2100次模型查询(对比此前Isabelle上SOTA Subgoal-XL使用16384次查询仅达56.1%)。我们还实现了Lean 4版本的ProofAug,使Kimina-Prover-Preview-Distill-1.5B在miniF2F-test上的pass@1性能从44.3%提升至50.4%。代码已开源。

原文摘要 · Abstract (English)

The synergy between deep learning models and traditional automation tools, such as built-in tactics of the proof assistant and off-the-shelf automated theorem provers, plays a crucial role in developing robust and efficient neural theorem provers(NTPs). However, for proof synthesis with LLMs, previous work applies automation tools either only when explicitly invoked by the model or at a single granularity level, failing to fully exploit their power. To solve this issue, we propose ProofAug, a procedure that equips LLMs with automation methods at various granularities through fine-grained structure analysis of model-generated proof proposals. ProofAug also serves as a versatile plug-and-play module that seamlessly integrates with any tree-search algorithm, enabling our construction of an efficient recursive proving (ERP) module to further enhance performance. The superiority of our method is validated on the miniF2F benchmark using the open-source deepseek-math-7b-base model and the Isabelle proof assistant. Notably, by additionally employing a mixed prompting strategy, we achieve a cumulative pass rate of 66.0% after curation of the dataset (61.9% for the original version) with 2100 queries to the model per problem (In contrast, the previous SOTA in Isabelle, Subgoal-XL, only achieves 56.1% using 16384 queries per problem). We also implement a Lean 4 version of ProofAug that can improve the pass@1 performance of Kimina-Prover-Preview-Distill-1.5B from 44.3% to 50.4% on miniF2F-test. Our code is available at https://github.com/haoxiongliu/ProofAug.

定理证明大模型自动化推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。