用离线强化学习提升代码生成大模型效率，省时省资源。
Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning
- 利用已有代码数据集进行离线强化学习训练
- 小模型和复杂编码任务中性能提升显著
- 适合资源有限或需快速迭代的代码生成场景
使用在线强化学习（RL）进行后训练是代码生成大模型的重要步骤。然而，在线RL需要模型推理与生成结果验证，耗时且资源开销大。本文探索利用现有代码数据集进行离线强化学习在代码生成模型中的应用。实验表明，离线强化学习是一种有效的提升大模型性能的策略，尤其对小型模型和复杂编码问题效果更佳。
原文摘要 · Abstract (English)
Post-training using online reinforcement learning (RL) is an important training step for LLMs, including code-generating models. However, online RL for code generation involves LLM inference and verification of the generated output, which can take considerable time and resources. In this paper, we explore the application of offline RL to code-generating models by leveraging existing code datasets. Our experiments demonstrate that offline RL is an effective training strategy for improving LLM performance. We show that offline RL can be especially beneficial for small LLMs and challenging coding problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。