让大模型准确判断定理能否使用,避免错误引用。
Reliable Use of Lemmas via Eligibility Reasoning and Section$-$Aware Reinforcement Learning
- 分两段输出检查前提和结论有效性,提升判断可靠性。
- 在扰动测试中表现更好,能识别被破坏的适用条件。
- 适合需要严谨推理的数学证明场景,如竞赛或形式化验证。
当前大型语言模型在数学基准上表现强劲,但常错误应用定理,未验证前提就引入结论。本文将定理判断建模为结构化预测任务:给定命题与候选定理,模型需输出前提检查和结论效用检查,据此得出有用性判断。提出RULES方法,通过双段输出编码该规范,并采用强化学习结合章节感知损失掩码,对出错部分施加惩罚。训练与评估基于多样化的自然语言与形式化证明语料;鲁棒性通过保留测试集扰动套件评估;端到端测试涵盖竞赛风格、扰动对齐及定理驱动问题,覆盖多种LLM。结果表明,在域内任务中持续优于基础模型与单标签强化学习基线,对破坏适用性的扰动改进更显著,端到端任务达到相当或小幅提升;消融实验显示双段输出与章节感知强化学习均为实现鲁棒性的必要组件。
原文摘要 · Abstract (English)
Recent large language models (LLMs) perform strongly on mathematical benchmarks yet often misapply lemmas, importing conclusions without validating assumptions. We formalize lemma$-$judging as a structured prediction task: given a statement and a candidate lemma, the model must output a precondition check and a conclusion$-$utility check, from which a usefulness decision is derived. We present RULES, which encodes this specification via a two$-$section output and trains with reinforcement learning plus section$-$aware loss masking to assign penalty to the section responsible for errors. Training and evaluation draw on diverse natural language and formal proof corpora; robustness is assessed with a held$-$out perturbation suite; and end$-$to$-$end evaluation spans competition$-$style, perturbation$-$aligned, and theorem$-$based problems across various LLMs. Results show consistent in$-$domain gains over both a vanilla model and a single$-$label RL baseline, larger improvements on applicability$-$breaking perturbations, and parity or modest gains on end$-$to$-$end tasks; ablations indicate that the two$-$section outputs and section$-$aware reinforcement are both necessary for robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。