让大模型学会生成可验证的反例,补全数学推理短板
Learning to Disprove: Formal Counterexample Generation with Large Language Models
- 用符号变异法自动构造反例训练数据
- 在三个新基准上实现显著性能提升
- 适合需要严谨数学验证的研究者
数学推理需要两大核心能力:为真命题构造严格证明,以及为假命题发现反例。然而当前人工智能在数学领域主要聚焦于证明生成,忽视了反例发现这一同样关键的任务。本文通过微调大语言模型(LLMs)来推理并生成反例,将该任务形式化为形式化反例生成,要求模型不仅提出候选反例,还需生成可在Lean 4定理证明器中自动验证的正式证明。为支持有效学习,我们引入一种符号变异策略,通过系统性地提取定理并剔除部分假设,生成多样化的反例实例。结合精心构建的数据集,该策略支持多奖励专家迭代框架,显著提升模型在反例生成和定理证明任务上的效率与效果。在三个新收集的基准测试上,实验验证了该方法的优势,表明变异策略与训练框架带来了显著性能提升。
原文摘要 · Abstract (English)
Mathematical reasoning demands two critical, complementary skills: constructing rigorous proofs for true statements and discovering counterexamples that disprove false ones. However, current AI efforts in mathematics focus almost exclusively on proof construction, often neglecting the equally important task of finding counterexamples. In this paper, we address this gap by fine-tuning large language models (LLMs) to reason about and generate counterexamples. We formalize this task as formal counterexample generation, which requires LLMs not only to propose candidate counterexamples but also to produce formal proofs that can be automatically verified in the Lean 4 theorem prover. To enable effective learning, we introduce a symbolic mutation strategy that synthesizes diverse training data by systematically extracting theorems and discarding selected hypotheses, thereby producing diverse counterexample instances. Together with curated datasets, this strategy enables a multi-reward expert iteration framework that substantially enhances both the effectiveness and efficiency of training LLMs for counterexample generation and theorem proving. Experiments on three newly collected benchmarks validate the advantages of our approach, showing that the mutation strategy and training framework yield significant performance gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。