用迭代推理与回溯验证，让大模型解数学题准确率翻倍
BEATS: Optimizing LLM Mathematical Capabilities with BackVerify and Adaptive Disambiguate based Efficient Tree Search
- 设计新提示引导模型分步重写和推进解题
- 在MATH基准上将Qwen2-7b得分从36.94提升至61.52
- 适合想提升数学推理能力的研究者和开发者
大型语言模型在诸多任务中表现优异，但在数学问题求解上仍面临挑战，因数学要求严格逻辑。此前研究通过监督微调、提示工程及基于搜索的方法改进模型数学能力，但效果有限且计算开销大。本文提出BEATS方法，利用新设计的提示引导模型分步重写、逐步推进并生成答案。引入基于LLM的回溯验证机制，确保解题正确性，并采用剪枝树搜索优化搜索效率。实验显示，该方法使Qwen2-7b-Instruct在MATH基准上的得分从36.94提升至61.52，超越GPT4的42.5分。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have exhibited exceptional performance across a broad range of tasks and domains. However, they still encounter difficulties in solving mathematical problems due to the rigorous and logical nature of mathematics. Previous studies have employed techniques such as supervised fine-tuning (SFT), prompt engineering, and search-based methods to improve the mathematical problem-solving abilities of LLMs. Despite these efforts, their performance remains suboptimal and demands substantial computational resources. To address this issue, we propose a novel approach, BEATS, to enhance mathematical problem-solving abilities. Our method leverages newly designed prompts that guide the model to iteratively rewrite, advance by one step, and generate answers based on previous steps. Additionally, we introduce a new back-verification technique that uses LLMs to validate the correctness of the generated answers. Furthermore, we employ a pruning tree search to optimize search time while achieving strong performance. Notably, our method improves Qwen2-7b-Instruct's score from 36.94 to 61.52, outperforming GPT4's 42.5 on the MATH benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。