用知识检索提升代码生成中间步骤评估，减少错误与计算开销。
RPM-MCTS: Knowledge-Retrieval as Process Reward Model with Monte Carlo Tree Search for Code Generation
- 用知识库检索替代训练复杂过程奖励模型，评估中间算法步骤。
- 在4个基准上超越现有方法，节省约15%的生成令牌消耗。
- 适合需要高效、准确生成代码的研究者与开发者使用。
基于树搜索的方法在提升大语言模型代码生成能力方面取得显著进展。然而，由于难以有效评估中间算法步骤，且无法及时定位和纠正错误步骤，这些方法常生成错误代码并带来更高的计算成本。为解决上述问题，我们提出RPM-MCTS，一种利用知识库检索作为过程奖励模型、结合蒙特卡洛树搜索的方法，用于评估中间算法步骤。通过知识库检索，RPM-MCTS避免了复杂的过程奖励模型训练。在扩展阶段，采用相似性过滤去除冗余节点，保障推理路径多样性。此外，方法利用沙盒执行反馈定位生成过程中的错误步骤，实现及时、精准的修正。在四个公开代码生成基准上的大量实验表明，RPM-MCTS优于当前最先进方法，同时实现约15%的令牌消耗降低。此外，使用RPM-MCTS构建的数据对基础模型进行全微调，显著提升了其代码生成能力。
原文摘要 · Abstract (English)
Tree search-based methods have made significant progress in enhancing the code generation capabilities of large language models. However, due to the difficulty in effectively evaluating intermediate algorithmic steps and the inability to locate and timely correct erroneous steps, these methods often generate incorrect code and incur increased computational costs. To tackle these problems, we propose RPM-MCTS, an effective method that utilizes Knowledge-Retrieval as Process Reward Model based on Monte Carlo Tree Search to evaluate intermediate algorithmic steps. By utilizing knowledge base retrieval, RPM-MCTS avoids the complex training of process reward models. During the expansion phase, similarity filtering is employed to remove redundant nodes, ensuring diversity in reasoning paths. Furthermore, our method utilizes sandbox execution feedback to locate erroneous algorithmic steps during generation, enabling timely and targeted corrections. Extensive experiments on four public code generation benchmarks demonstrate that RPM-MCTS outperforms current state-of-the-art methods while achieving an approximately 15% reduction in token consumption. Furthermore, full fine-tuning of the base model using the data constructed by RPM-MCTS significantly enhances its code capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。