用强化学习让评阅模型主动改进数学形式化,提升准确性
CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization
- 设计可学习的评阅模型,主动指导形式化生成
- 在28.5万题数据集上显著超越现有基线模型
- 适合研究自动化定理证明与形式化推理的学者
将自然语言数学命题转化为可执行的形式化代码是自动定理证明中的核心挑战。以往工作聚焦于生成与编译成功率,却忽视了评阅阶段——即评估生成形式化是否准确捕捉原始问题语义。本文提出CriticLean,一种新型评阅引导的强化学习框架,将评阅器从被动验证者升级为主动学习组件。首先,我们训练CriticLeanGPT,通过监督微调与强化学习,严格评估Lean 4形式化的语义保真度;其次,构建CriticLeanBench基准,衡量模型区分语义正确与错误形式化的能力,并证明训练后的CriticLeanGPT显著优于强开源与闭源基线。基于此框架,我们构建FineLeanCorpus,一个包含超过28.5万道题、覆盖丰富领域与难度、经人工评估高准确率的数据集。结果表明,优化评阅阶段对生成可靠形式化至关重要,CriticLean为未来形式化数学推理研究提供重要启示。
原文摘要 · Abstract (English)
Translating natural language mathematical statements into formal, executable code is a fundamental challenge in automated theorem proving. While prior work has focused on generation and compilation success, little attention has been paid to the critic phase-the evaluation of whether generated formalizations truly capture the semantic intent of the original problem. In this paper, we introduce CriticLean, a novel critic-guided reinforcement learning framework that elevates the role of the critic from a passive validator to an active learning component. Specifically, first, we propose the CriticLeanGPT, trained via supervised fine-tuning and reinforcement learning, to rigorously assess the semantic fidelity of Lean 4 formalizations. Then, we introduce CriticLeanBench, a benchmark designed to measure models' ability to distinguish semantically correct from incorrect formalizations, and demonstrate that our trained CriticLeanGPT models can significantly outperform strong open- and closed-source baselines. Building on the CriticLean framework, we construct FineLeanCorpus, a dataset comprising over 285K problems that exhibits rich domain diversity, broad difficulty coverage, and high correctness based on human evaluation. Overall, our findings highlight that optimizing the critic phase is essential for producing reliable formalizations, and we hope our CriticLean will provide valuable insights for future advances in formal mathematical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。