提升代码生成的正确率与运行效率，让小模型也能达到大模型水平。
Towards Better Correctness and Efficiency in Code Generation
- 用强化学习框架优化代码效率，设计新型性能奖励机制。
- 在7B模型上提升正确率10.18%、运行效率7.75%，接近更大模型表现。
- 两阶段调优策略兼顾准确性和效率，适合性能敏感场景应用。
尽管代码大模型在代码生成方面取得了显著进展，但生成代码常存在运行效率低的问题，限制了其在性能敏感场景中的实际应用。为此，我们提出一种面向效率的强化学习框架，采用新型性能奖励机制。基于该框架，我们深入分析代码效率问题，识别并提出了克服关键瓶颈的方法：(1) 动态探索克服离线微调的数据静态性，发现更高效的代码实现；(2) 无错误敏感的强化学习方法与高对比度效率信号对缓解系统性错误、实现有效优化至关重要；(3) 在高正确率基线基础上进行在线探索最有效，可在不牺牲准确性前提下提升效率。结合这些发现，我们最终提出两阶段调优方法，在7B模型上实现正确率提升10.18%、运行效率提升7.75%，性能媲美更大模型。
原文摘要 · Abstract (English)
While code large language models have demonstrated remarkable progress in code generation, the generated code often exhibits poor runtime efficiency, limiting its practical application in performance-sensitive scenarios. To address this limitation, we propose an efficiency-oriented reinforcement learning framework guided by a novel performance reward. Based on this framework, we take a deeper dive into the code efficiency problem, identifying then proposing methods to overcome key bottlenecks: (1) Dynamic exploration overcomes the static data constraints of offline fine-tuning, enabling the discovery of more efficient code implementations. (2) The error-insensitive reinforcement learning method and high-contrast efficiency signals are crucial for mitigating systematic errors and achieving effective optimization. (3) Online exploration is most effective when starting from a high-correctness baseline, as this allows for efficiency improvements without sacrificing accuracy. With these discoveries, we finally propose a two-stage tuning method, which achieves high and balanced performance across correctness and efficiency. The results of experiments show the effectiveness of the method, which improves code correctness by 10.18\% and runtime efficiency by 7.75\% on a 7B model, achieving performance comparable to much larger model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。