用强化学习提升小语种编程代码生成准确率
From Reasoning to Code: GRPO Optimization for Underrepresented Languages
- 结合小模型与GRPO算法，通过推理优化代码生成
- 在GSM8K上实现推理质量与代码准确率显著提升
- 适合资源匮乏编程语言的开发者使用
使用大型语言模型（LLMs）生成准确且可执行的代码，对缺乏公共训练数据的低资源编程语言（如Prolog、Lisp）仍具挑战性。本文提出一种通用的强化学习方法，将小规模Qwen2.5-Coder模型与组相对策略优化（GRPO）结合，通过推理实现有效代码生成。为应对数据稀疏问题，将执行反馈直接融入强化学习循环，采用同时考虑逻辑正确性与结构格式的奖励机制。在GSM8K数据集上的实验表明，该方法在低资源语言中显著提升了推理质量和代码准确性。结果表明，该方法通过符号推理与解释器反馈，有望为大量缺乏训练资源的编程语言提供支持。
原文摘要 · Abstract (English)
Generating accurate and executable code using Large Language Models (LLMs) remains a significant challenge for underrepresented programming languages, such as Prolog and Lisp, due to the scarcity of public training data compared to high-resource languages like Python. This paper introduces a generalizable Reinforcement Learning (RL) approach that combines small-scale versions of the Qwen2.5-Coder model with Group Relative Policy Optimization (GRPO) to enable effective code generation through reasoning. To address the limitations of sparse datasets, we integrate execution-driven feedback directly into the RL loop, utilizing a reward system that exploits both logical correctness and structural formatting. Experimental results on GSM8K dataset demonstrate significant improvements in reasoning quality and code accuracy across underrepresented languages. These findings underscore the potential of our approach to benefit a wide range of programming languages lacking extensive training resources by leveraging symbolic reasoning and interpreter-based feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。