用约束搜索提升大模型数学推理能力，零样本下7B模型超72B基线。
CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Model
- 引入约束动作集与部分序规则，提升状态空间多样性与动作合理性。
- 零样本下7B模型准确率达83.4%，超越72B基线4.8个百分点。
- 适合追求高效数学推理的模型优化研究者参考。
本文提出约束蒙特卡洛树搜索（CMCTS）框架，以增强大语言模型（LLM）的数学推理能力。通过引入约束动作空间、过程奖励模型（PRM）和部分序规则，有效解决现有MCTS方法在状态空间多样性和动作选择合理性方面的局限。具体而言，在扩展阶段，CMCTS将动作采样限制在预定义的约束动作集中，以增加候选状态多样性；在模拟阶段，引入部分序规则和PRM优化动作选择，防止不合理状态转移。实验结果表明，CMCTS在多个数学推理基准上表现优异。在零样本设置下，7B参数模型平均准确率达83.4%，超越72B基线模型4.8个百分点。消融实验证明框架各组件均对性能提升至关重要，协同使用可充分发挥各自优势。整体上，CMCTS为增强LLM数学推理能力提供了有效路径，并具备理论支持，为未来推理任务提供新思路。
原文摘要 · Abstract (English)
This paper introduces the Constrained Monte Carlo Tree Search (CMCTS) framework to enhance the mathematical reasoning capabilities of Large Language Models (LLM). By incorporating a constrained action space, Process Reward Model (PRM), and partial order rules, CMCTS effectively addresses the limitations of existing MCTS methods in terms of state space diversity and action selection rationality. Specifically, during the expansion phase, CMCTS restricts action sampling to a predefined constrained action set to increase candidate state diversity. In the simulation phase, it introduces partial order rules and PRM to optimize action selection and prevent unreasonable state transitions. Experimental results show that CMCTS performs outstandingly across multiple mathematical reasoning benchmarks. Under a zero-shot setting, a 7B-parameter model achieves an average accuracy of 83.4\%, surpassing the 72B baseline model by 4.8\%. Ablation studies demonstrate that each component of the framework is crucial for performance improvement, and their combined use fully leverages their respective strengths. Overall, the CMCTS framework provides an effective approach to enhancing LLM mathematical reasoning capabilities, supported by theoretical analysis, and offers novel insights for future reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。