arXiv:2601.18944cs.AIcs.PL2026-01中稿 · ICLR被引 6

首个面向程序验证条件的神经定理证明基准,评估大模型在真实代码中的推理能力。

Neural Theorem Proving for Verification Conditions: A Real-World Benchmark

  • 构建跨语言真实项目验证条件数据集,支持Isabelle、Lean等系统。
  • 大模型在工业级验证条件上成功率不足30%,仍需大幅改进。
  • 适合研究形式化验证与AI推理融合的学者及工业验证团队。

定理证明是程序验证的核心,但自动化证明验证条件(VCs)仍是主要瓶颈。现实中的程序验证常遇到现有自动定理证明器无法解决的难题,导致大量手动证明,严重影响实际应用。尽管神经定理证明(NTP)在数学竞赛中表现优异,展示了机器学习在形式推理中的潜力,但在程序验证特别是验证条件证明方面的应用仍基本空白。现有工作虽涉及注释生成和相关定理证明,但缺乏专门针对此核心挑战的基准。本文提出首个真实世界多语言验证条件神经定理证明基准NTP4VC,基于Linux与Contiki-OS内核等真实项目,通过Why3与Frama-C工业管道,在Isabelle、Lean与Rocq等正式语言间生成语义等价测试用例。我们评估了通用及微调过的大型语言模型(LLMs)在该基准上的表现,结果显示尽管大模型展现出一定潜力,但在真实验证条件证明上仍面临显著挑战,揭示出巨大研究差距与机遇。

原文摘要 · Abstract (English)

Theorem proving is fundamental to program verification, where the automated proof of Verification Conditions (VCs) remains a primary bottleneck. Real-world program verification frequently encounters hard VCs that existing Automated Theorem Provers (ATPs) cannot prove, leading to a critical need for extensive manual proofs that burden practical application. While Neural Theorem Proving (NTP) has achieved significant success in mathematical competitions, demonstrating the potential of machine learning approaches to formal reasoning, its application to program verification--particularly VC proving--remains largely unexplored. Despite existing work on annotation synthesis and verification-related theorem proving, no benchmark has specifically targeted this fundamental bottleneck: automated VC proving. This work introduces Neural Theorem Proving for Verification Conditions (NTP4VC), presenting the first real-world multi-language benchmark for this task. From real-world projects such as Linux and Contiki-OS kernel, our benchmark leverages industrial pipelines (Why3 and Frama-C) to generate semantically equivalent test cases across formal languages of Isabelle, Lean, and Rocq. We evaluate large language models (LLMs), both general-purpose and those fine-tuned for theorem proving, on NTP4VC. Results indicate that although LLMs show promise in VC proving, significant challenges remain for program verification, highlighting a large gap and opportunity for future research.

定理证明程序验证大模型形式化方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。