arXiv:2608.14221cs.AIcs.CL2026-08

用知识检索与验证反馈提升数学自动形式化准确率

MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval and Verification-Guided Refinement

论文配图:MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval and Verification-Guided Refinement
图 1 · 摘自论文原文
  • 先检索Mathlib知识再生成,引导模型精准匹配数学定义
  • 通过编译诊断和语义一致性反馈迭代修正,生成36.7万条验证数据
  • 80亿参数模型在多个基准上超越320亿参数专用模型

自动形式化通常被视作将自然语言数学命题翻译为可机器验证的形式语言(如Lean 4)。然而,忠实的形式化不仅需翻译,还需将数学概念映射到Mathlib等正式库中复杂的类型与定义层级,并确保生成语句保留原命题语义。现有方法因过度依赖模型参数记忆库内知识,且数据构建常采用单次输出过滤,缺乏反馈修订机制而表现受限。为此,我们提出MathForm框架,通过Mathlib知识检索与验证引导的迭代精炼,构建可验证训练数据。生成前,检索规划器从Mathlib获取相关定义与已有形式化以指导生成;生成后,利用编译诊断与语义一致性反馈进行修正。基于此框架,我们构建了FormalVerse数据集,包含约36.7万条跨多个数学领域的已验证实例。随后,我们使用监督微调结合强化学习训练MathForm-8B。在六个基准测试中,MathForm-8B在语法检查(SC)下平均通过率88.06%,在一致性检查(CC)下达72.37%,优于多个320亿参数专用模型。在挑战性子集FATE-H与FATE-X上,其CC通过率分别为63%与37%,均超过最强基线。

原文摘要 · Abstract (English)

Autoformalization is commonly framed as translating natural-language mathematical statements into machine-verifiable formal languages such as Lean 4. However, faithful formalization requires more than translation. Models must map mathematical concepts to the complex hierarchy of types and definitions in formal libraries such as Mathlib, while ensuring that generated statements preserve the meaning of the source propositions. Existing approaches struggle because they rely heavily on the model's parametric memory for library-specific knowledge, while common data construction pipelines often resort to filtering single-pass outputs and lack mechanisms for feedback-driven revision. To address these challenges, we introduce MathForm, an autoformalization framework for constructing verified training data through Mathlib knowledge retrieval and verification-guided iterative refinement. Before generation, a retrieval planner gathers relevant definitions and existing formalizations from Mathlib to guide the formalization generator. Generated statements are then revised using compiler diagnostics and semantic-consistency feedback. Using this framework, we construct FormalVerse, a Lean 4 dataset containing approximately 367K verified examples across diverse mathematical domains and sources. We then train MathForm-8B through supervised fine-tuning followed by reinforcement learning. Across six benchmarks, MathForm-8B achieves average Pass@8 rates of 88.06% under Syntax Check (SC) and 72.37% under Consistency Check (CC), outperforming multiple specialized 32B autoformalizers. On the challenging FATE-H and FATE-X subsets, it attains CC pass rates of 63% and 37%, exceeding the strongest specialized baselines in both cases.

数学形式化知识检索验证反馈Lean 4

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。