用递归分解让大模型自动生成可验证的数学证明。
Hilbert: Recursively Building Formal Proofs with Informal Reasoning
- 结合自然语言推理与形式化验证,递归拆解问题求解。
- 在miniF2F上达99.2%准确率,优于现有公开方法6.6个百分点。
- 适合研究形式化证明与智能推理的学者使用。
大型语言模型(LLMs)展现出强大的数学推理能力,但其解题过程常含错误且难以自动验证。形式化定理证明系统(如Lean 4)可实现完全准确的验证,推动了专用证明者LLM的发展。然而,当前证明者LLM解决问题的数量远低于通用型自然语言LLM。我们提出Hilbert,一个融合非正式推理与形式化验证优势的代理框架。系统协调四个组件:擅长数学推理的非正式LLM、优化于Lean 4策略的专用证明者LLM、形式化验证器及语义定理检索器。当证明者无法解决时,Hilbert通过递归分解将问题拆分为子目标,并由证明者或推理器处理。利用验证反馈修正错误证明。实验显示,Hilbert在关键基准上显著优于现有方法:在miniF2F上达到99.2%,较最佳公开方法高出6.6个百分点;在PutnamBench上解决462/660题(70.0%),较种子模型SeedProver(50.4%)提升422%。该成果有效缩小了非正式推理与形式化证明之间的差距。代码已开源:https://github.com/Rose-STL-Lab/ml-hilbert。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate impressive mathematical reasoning abilities, but their solutions frequently contain errors that cannot be automatically checked. Formal theorem proving systems such as Lean 4 offer automated verification with complete accuracy, motivating recent efforts to build specialized prover LLMs that generate verifiable proofs in formal languages. However, a significant gap remains: current prover LLMs solve substantially fewer problems than general-purpose LLMs operating in natural language. We introduce Hilbert, an agentic framework that bridges this gap by combining the complementary strengths of informal reasoning and formal verification. Our system orchestrates four components: an informal LLM that excels at mathematical reasoning, a specialized prover LLM optimized for Lean 4 tactics, a formal verifier, and a semantic theorem retriever. Given a problem that the prover is unable to solve, Hilbert employs recursive decomposition to split the problem into subgoals that it solves with the prover or reasoner LLM. It leverages verifier feedback to refine incorrect proofs as necessary. Experimental results demonstrate that Hilbert substantially outperforms existing approaches on key benchmarks, achieving 99.2\% on miniF2F, 6.6\% points above the best publicly available method. Hilbert achieves the \textbf{strongest known result} from a publicly available model on PutnamBench. It solves 462/660 problems (70.0\%), outperforming proprietary approaches like SeedProver (50.4\%) and achieving a 422\% improvement over the best publicly available baseline. Thus, Hilbert effectively narrows the gap between informal reasoning and formal proof generation. Code is available at https://github.com/Rose-STL-Lab/ml-hilbert.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。