arXiv:2411.11053cs.CLcs.AI2024-11IJCAI被引 13

用自驱动推理增强代码生成,让小模型也能解决复杂问题。

SRA-MCTS: Self-driven Reasoning Augmentation with Monte Carlo Tree Search for Code Generation

  • 用蒙特卡洛树搜索自动生成高质量推理路径
  • 无需额外监督,小模型在复杂任务上成功率显著提升
  • 适合想提升代码生成能力的研究者和开发者

大型语言模型在简单代码生成任务中表现优异,但在处理复杂问题时仍面临推理与分解能力不足的挑战。为此,我们提出自驱动推理增强方法 SRA-MCTS,通过模型自身生成高质量的中间推理路径,形成正向反馈循环,实现持续优化。该方法完全由模型自主完成,无需额外监督信号。通过融合自然语言推理路径并转化为可执行代码,确保分析准确性,显著提升复杂任务求解成功率。实验表明,即使不依赖外部监督,该方法在不同规模模型上均取得性能提升,展现了小模型自我改进的巨大潜力。相较于传统链式思维(CoT)方法,SRA-MCTS在性能下降场景下仍保持稳定,且在 pass@10 等多样性指标上表现更优。我们鼓励进一步探索训练数据中的推理过程,以增强模型应对复杂问题的能力。代码与数据已公开于 https://github.com/DIRECT-BIT/SRA-MCTS。

原文摘要 · Abstract (English)

Large language models demonstrate exceptional performance in simple code generation tasks but still face challenges in tackling complex problems. These challenges may stem from insufficient reasoning and problem decomposition capabilities. To address this issue, we propose a reasoning-augmented data generation process, SRA-MCTS, which guides the model to autonomously generate high-quality intermediate reasoning paths. This creates a positive feedback loop, enabling continuous improvement. Our method operates entirely through the model itself without requiring additional supervision. By synthesizing natural language reasoning paths and translating them into executable code, the approach ensures analytical accuracy and enhances the success rate in solving complex tasks. Experimental results show that, even without additional supervisory signals, our method achieves performance improvements across different model scales, demonstrating the significant potential of self-improvement in small models. Furthermore, the method remains robust when traditional Chain-of-Thought (CoT) approaches exhibit performance degradation, with notable improvements observed in diversity metrics such as pass@10. We encourage further exploration of reasoning processes within training data to enhance the ability of language models to address complex problems. Our code and data are public at https://github.com/DIRECT-BIT/SRA-MCTS.

代码生成推理增强自进化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。