用多语言生成结果集成,提升代码模型准确率。
Multi-Programming Language Ensemble for Code Generation in Large Language Model
- 让模型同时生成多种语言代码,再融合优化结果
- 在HumanEval上达到96.25%准确率,比基线高17.92%
- 适合想提升代码生成鲁棒性的研究者和开发者
大型语言模型(LLMs)在单次代码生成任务中表现显著提升,但现有方法大多仅聚焦于单一编程语言的代码生成,忽视了利用LLM多语言能力的潜力。不同编程语言中,模型错误模式存在差异,因此通过融合多语言输出可实现更稳健的性能提升。本文提出多编程语言集成方法(MPLE),将每种语言的代码生成视为一个‘弱专家’,通过有效整合其输出,缓解语言特定的错误与偏差。该策略充分利用各语言间的互补优势,使模型生成更准确、更鲁棒的代码。该方法可无缝集成反射算法和蒙特卡洛树搜索等技术,进一步提升生成质量。实验表明,该框架在现有基准(HumanEval 和 HumanEval-plus)上,性能提升最高达17.92%,在HumanEval上取得96.25%的准确率,超越多种LLM模型的当前最优水平。代码将开源至 https://github.com/NinjaTech-AI/MPLE。
原文摘要 · Abstract (English)
Large language models (LLMs) have significantly improved code generation, particularly in one-pass code generation. However, most existing approaches focus solely on generating code in a single programming language, overlooking the potential of leveraging the multi-language capabilities of LLMs. LLMs have varying patterns of errors across different languages, suggesting that a more robust approach could be developed by leveraging these multi-language outputs. In this study, we propose Multi-Programming Language Ensemble (MPLE), a novel ensemble-based method that utilizes code generation across multiple programming languages to enhance overall performance. By treating each language-specific code generation process as an individual "weak expert" and effectively integrating their outputs, our method mitigates language-specific errors and biases. This multi-language ensemble strategy leverages the complementary strengths of different programming languages, enabling the model to produce more accurate and robust code. Our approach can be seamlessly integrated with commonly used techniques such as the reflection algorithm and Monte Carlo tree search to improve code generation quality further. Experimental results show that our framework consistently enhances baseline performance by up to 17.92% on existing benchmarks (HumanEval and HumanEval-plus), with a standout result of 96.25% accuracy on the HumanEval benchmark, achieving new state-of-the-art results across various LLM models. The code will be released at https://github.com/NinjaTech-AI/MPLE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。