让大模型自己检查代码错误并修正,不依赖外部工具就能自动优化编程结果。
ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

- 用强化学习教会模型在生成代码后自我反思、发现漏洞并修正。
- 单次尝试下在多个基准测试中达到94.5%准确率,超越多数开源模型。
- 推理效率更高,节省约40%计算开销,适合实际部署使用。
尽管大语言模型已革新代码生成,但传统单步生成方法在复杂算法任务上表现受限。现有迭代优化方法多依赖外部反馈或执行引擎,成本高且不自主。本文提出ReflexiCoder,一种基于强化学习的框架,将生成、反思、纠错的完整推理过程内化到模型权重中,实现无需外部依赖的自主自修正。通过细粒度奖励函数训练,模型学会在无真实反馈情况下自我调试。在七个基准上验证,ReflexiCoder-8B在HumanEval(Plus)达94.51%,MBPP(Plus)达81.80%,BigCodeBench为35.00%,LiveCodeBench为52.21%,CodeForces为37.34%,单次尝试即媲美甚至超过如GPT-5.1等闭源模型。该框架推理效率提升约40%,显著降低计算开销。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) have revolutionized code generation, standard ``System 1'' approaches that generate solutions in a single forward pass often hit a performance ceiling on complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model's weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-only training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B to 14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, efficient reasoning and reflection patterns. The source code and data are available at https://github.com/juyongjiang/ReflexiCoder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。