构建首个面向计算与应用数学的Lean形式化基准,填补大模型数学推理空白。
CAM-Bench: A Benchmark for Computational and Applied Mathematics in Lean

- 从教材习题重构局部定义与上下文,生成可验证的Lean证明目标
- 涵盖1000个优化、数值线性代数等应用数学问题,覆盖多类核心领域
- 适合评估大模型在复杂符号推理与长期证明控制中的表现
形式化定理证明基准能对大语言模型的数学推理能力进行机械验证。然而现有基准主要聚焦奥数题和代数领域,计算与应用数学仍被忽视。我们提出CAM-Bench,一个基于Lean 4的1000个计算与应用数学证明目标的基准,覆盖优化、数值线性代数和数值分析等领域。这些问题源自教材习题,常依赖局部定义、符号、算法和基础结论。为构建该基准,我们开发了依赖恢复流水线,重建每个问题所需的教材上下文,并将其规范化为独立的非正式定理后转化为Lean目标。通过Lean编译和语义审查验证结果的形式正确性与原始习题的语义一致性。每个问题均公开原始习题、恢复的上下文、规范化非正式定理及最终Lean目标。CAM-Bench补充了现有形式化数学基准,聚焦依赖教材概念与基础定理的应用数学问题,其中许多无法直接调用Mathlib4标准引理。我们在该基准上评估主流大语言模型与形式化代理,分析其在追踪局部假设、应用基础结论、分解证明及维持长程控制方面的常见失败模式。
原文摘要 · Abstract (English)
Formal theorem-proving benchmarks enable mechanically verifiable evaluation of mathematical reasoning in large language models. However, existing benchmarks mainly focus on Olympiad-style problems and algebraic domains, leaving computational and applied mathematics underrepresented. We introduce CAM-Bench, a Lean 4 theorem-proving benchmark of 1,000 Lean proof targets in computational and applied mathematics, with coverage spanning optimization, numerical linear algebra, and numerical analysis. These problems are adapted from textbook exercises and often depend on locally introduced definitions, notation, algorithms, and elementary results. To construct CAM-Bench, we develop a dependency-recovery pipeline that reconstructs the local textbook context needed to state each problem faithfully. It then normalizes each problem into a standalone informal theorem and translates it into a Lean target. We validate the resulting formal problems through Lean compilation and semantic review, checking both formal correctness and semantic alignment with the original exercises. For each problem, we release the raw exercise, recovered context, normalized informal theorem, and final Lean target. CAM-Bench complements existing formal mathematics benchmarks by targeting applied mathematics problems that rely on textbook concepts and elementary theorems, many of which are not directly available as standard Mathlib4 lemmas. We evaluate widely used large language models and formalization agents on CAM-Bench, and analyze common failure modes in tracking local assumptions, applying elementary results, decomposing proofs, and maintaining long-horizon control in Lean.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。