让大模型像走迷宫一样推理数学题，提升解题能力
Kwai-STaR: Transform LLMs into State-Transition Reasoners
- 把解题过程看作状态转移，设计专属状态空间
- 用生成的状态转移数据训练模型，在GSM8K上显著提效
- 方法高效易用，适合想提升数学推理的开发者
数学推理对大语言模型的认知能力构成重大挑战。尽管已有多种方法提升大模型的数学能力，但很少关注状态转移在推理中的价值。本文将数学问题求解定义为从初始未解状态到最终求解状态的迁移过程，提出Kwai-STaR框架，将大模型转化为状态转移推理器以增强其直觉推理能力。该方法包含三步：(1) 针对数学推理构建状态空间；(2) 基于状态空间生成状态转移数据；(3) 通过课程训练策略将原始大模型转换为状态转移推理器。实验验证了Kwai-STaR的有效性：在小规模Kwai-STaR数据集上训练后，Mistral-7B和LLaMA-3等通用大模型在GSM8K和GSM-Hard数据集上均取得显著性能提升。此外，基于状态转移的设计使Kwai-STaR具备出色的训练与推理效率。更多实验正在开展以验证其普适性。
原文摘要 · Abstract (English)
Mathematical reasoning presents a significant challenge to the cognitive capabilities of LLMs. Various methods have been proposed to enhance the mathematical ability of LLMs. However, few recognize the value of state transition for LLM reasoning. In this work, we define mathematical problem-solving as a process of transiting from an initial unsolved state to the final resolved state, and propose Kwai-STaR framework, which transforms LLMs into State-Transition Reasoners to improve their intuitive reasoning capabilities. Our approach comprises three main steps: (1) Define the state space tailored to the mathematical reasoning. (2) Generate state-transition data based on the state space. (3) Convert original LLMs into State-Transition Reasoners via a curricular training strategy. Our experiments validate the effectiveness of Kwai-STaR in enhancing mathematical reasoning: After training on the small-scale Kwai-STaR dataset, general LLMs, including Mistral-7B and LLaMA-3, achieve considerable performance gain on the GSM8K and GSM-Hard dataset. Additionally, the state transition-based design endows Kwai-STaR with remarkable training and inference efficiency. Further experiments are underway to establish the generality of Kwai-STaR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。