arXiv:2412.20227cs.CL2024-12中稿 · NAACL被引 4

通过改写数学题表达方式,提升大模型解题能力。

LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning

  • 用题目改写增加语言多样性,增强模型泛化能力。
  • 专用训练目标显著提升模型在四数据集上的数学推理准确率。
  • 适合需要数学推理的教育、科研场景使用。

大语言模型(LLMs)在自然语言处理任务中表现优异,但在数学推理方面仍面临挑战,因复杂问题求解需兼顾语言理解与数学思维。现有方法多依赖集成策略,且目标领域数据稀缺。本文提出一种新方法,通过问题改写策略丰富数学题的语言形式,提升模型泛化能力;同时设计专用训练目标,强化模型对数学概念与推理过程的理解。我们在四个数据集上对多种LLM进行实验,结果表明该方法能有效提升模型在数学推理任务中的表现。研究验证了该方法对推进大模型能力的重要意义及其在真实应用场景中的潜力。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have shown remarkable performance in various natural language processing tasks but face challenges in mathematical reasoning, where complex problem-solving requires both linguistic understanding and mathematical reasoning skills. Existing approaches to address this challenge often rely on ensemble methods and suffer from the problem of data scarcity in target domains. In this work, we present a novel method to enhance LLMs' capabilities in mathematical reasoning tasks. Motivated by the need to bridge this gap, our approach incorporates a question paraphrase strategy, which aims at diversifying the linguistic forms of mathematical questions to improve generalization. Additionally, specialized training objectives are employed to guide the model's learning process, focusing on enhancing its understanding of mathematical concepts and reasoning processes. We conduct experiments on four datasets using different LLMs, and demonstrate the effectiveness of our approach in improving LLMs' performance on mathematical reasoning tasks. Our findings underscore the significance of our methodology in the advancement of large language models and its potential implications for real-world applications that require mathematical reasoning abilities.

数学推理大模型训练优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。