通过自我纠错与外部评审，让代码大模型持续迭代优化。
RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation
- 用大模型当裁判和批评家，自动评估并改进自动生成的代码。
- 在多个代码生成基准上，性能持续提升且用更少数据达到更好效果。
- 适合想提升代码生成能力的研究者与开发者参考。
代码生成随着大语言模型的发展受到越来越多关注。许多研究通过合成代码指令数据并进行监督微调来构建强大的代码大模型，但这些方法受限于教师模型蒸馏，忽略了自我生成代码进行迭代优化的潜力。本文提出自适应批判精炼（ACR），使模型通过自生成代码和外部批判实现自我改进，而非直接模仿教师模型的输出。具体而言，ACR包含基于大模型作为裁判的综合评分系统，用于评估代码质量，并结合大模型作为批评家的有选择性批判策略，对低质量代码进行修正。我们通过迭代应用ACR构建了RefineCoder系列模型，在多个代码生成基准上实现了持续性能提升。与同规模基线相比，RefineCoder系列在使用更少数据的情况下仍能达到相当或更优的表现。
原文摘要 · Abstract (English)
Code generation has attracted increasing attention with the rise of Large Language Models (LLMs). Many studies have developed powerful code LLMs by synthesizing code-related instruction data and applying supervised fine-tuning. However, these methods are limited by teacher model distillation and ignore the potential of iterative refinement by self-generated code. In this paper, we propose Adaptive Critique Refinement (ACR), which enables the model to refine itself by self-generated code and external critique, rather than directly imitating the code responses of the teacher model. Concretely, ACR includes a composite scoring system with LLM-as-a-Judge to evaluate the quality of code responses and a selective critique strategy with LLM-as-a-Critic to critique self-generated low-quality code responses. We develop the RefineCoder series by iteratively applying ACR, achieving continuous performance improvement on multiple code generation benchmarks. Compared to the baselines of the same size, our proposed RefineCoder series can achieve comparable or even superior performance using less data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。