用数学算子增强大模型推理能力，自动生成可计算的解题步骤。
Enhancing Mathematical Reasoning in LLMs with Background Operators
- 基于数学谓词构建背景算子，生成可验证的逻辑解题路径。
- 5折交叉验证自训练使准确率达84.6%（验证集）和84.8%（测试集）。
- 适合需要可解释数学推理的场景，如教育、科学计算。
我们提出利用背景算子提升大语言模型（LLM）的数学推理能力。为此，定义一组基础数学谓词作为基本构建块。针对每道数学题，开发包含问题特有谓词与由背景算子推导出的中间谓词的Prolog解决方案，确保所有解严格遵循预定义算子集。我们构建了MATH-Prolog语料库，源自MATH语料库中的计数与概率类别。为高效数据增强，采用5折交叉验证的自训练方法：逐轮生成新Prolog解，将验证正确的解加入训练集，持续迭代。实验表明，该方法在交叉验证集上准确率达84.6%，微调Meta-Llama-3.1-8B-Instruct模型后测试集准确率达84.8%。该方法成功发现对未见问题具有完整可计算推理步骤的新解。此外，在提示中引入背景数学谓词可提升解覆盖范围。
原文摘要 · Abstract (English)
We propose utilizing background operators for mathematical reasoning in large language models (LLMs). To achieve this, we define a set of fundamental mathematical predicates as the basic building blocks. For each mathematical problem, we develop a Prolog solution that includes problem-specific predicates and intermediate predicates derived from these background operators, ensuring that each solution adheres to the defined operator set. We introduce the MATH-Prolog corpus, which is derived from the counting and probability categories of the MATH corpus. For efficient data augmentation, we apply K-fold cross-validated self-training. This method incrementally generates new Prolog solutions for each fold, incorporating those verified as correct into the training set throughout the model training process. Our experimental results demonstrate that 5-fold crossvalidated self-training effectively identifies new, accurate Prolog solutions, achieving an accuracy of 84.6% on the cross-validated set, and 84.8% on the test set during fine-tuning the Meta-Llama-3.1-8B-Instruct model. This approach successfully uncovers new solutions with fully computable inference steps for previously unseen problems. Additionally, incorporating the background mathematical predicates into the prompt enhances solution coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。