小模型也能自我纠错写代码，效果逼近大模型。
Self-Correcting Code Generation Using Small Language Models
- 用在线强化学习让小模型逐步修正代码错误。
- 10亿参数模型在MBPP上提升35.8%，HumanEval上提升27.7%。
- 适合资源有限但需高可靠代码生成的场景。
自纠正已在代码生成中展现潜力，使语言模型通过多轮迭代优化输出。近期研究采用提示工程结合专有模型或训练方法来实现验证与反馈，但小型模型是否具备通过自我反思改进输出的能力仍不清楚。我们的研究发现，小模型在两种自纠正范式下均难以表现出有效的反思修正行为。为此，我们提出CoCoS，一种增强小语言模型多轮代码纠错能力的方法。该方法引入在线强化学习目标，使模型在多轮中自信保留正确部分，逐步修正错误部分。其累积奖励函数整合整个修正轨迹的奖励，并设计细粒度奖励以适应多轮纠错场景。这使得模型在初始输出基础上持续优化，显著提升质量。使用10亿参数模型，CoCoS在MBPP上较基线提升35.8%，在HumanEval上提升27.7%。
原文摘要 · Abstract (English)
Self-correction has demonstrated potential in code generation by allowing language models to revise and improve their outputs through successive refinement. Recent studies have explored prompting-based strategies that incorporate verification or feedback loops using proprietary models, as well as training-based methods that leverage their strong reasoning capabilities. However, whether smaller models possess the capacity to effectively guide their outputs through self-reflection remains unexplored. Our findings reveal that smaller models struggle to exhibit reflective revision behavior across both self-correction paradigms. In response, we introduce CoCoS, an approach designed to enhance the ability of small language models for multi-turn code correction. Specifically, we propose an online reinforcement learning objective that trains the model to confidently maintain correct outputs while progressively correcting incorrect outputs as turns proceed. Our approach features an accumulated reward function that aggregates rewards across the entire trajectory and a fine-grained reward better suited to multi-turn correction scenarios. This facilitates the model in enhancing initial response quality while achieving substantial improvements through self-correction. With 1B-scale models, CoCoS achieves improvements of 35.8% on the MBPP and 27.7% on HumanEval compared to the baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。