用强化学习让大模型生成更准确的游戏描述
Grammar and Gameplay-aligned RL for Game Description Generation with LLMs
- 结合语法和游戏概念奖励，双重优化生成质量
- 两阶段训练：先监督微调，再强化学习提升表现
- 适合需要高精度游戏描述生成的研究与开发
游戏描述生成（GDG）任务是将自然语言文本转换为游戏描述语言（GDL）中的游戏描述。以往研究利用大语言模型（LLMs）的上下文理解能力进行生成，但准确还原游戏特征仍具挑战。本文提出基于强化学习的LLM微调方法（RLGDG），通过引入语法奖励和概念奖励，同时提升生成结果的语法正确性和游戏概念一致性。采用两阶段训练策略：先进行监督微调（SFT），再应用强化学习（RL）。实验表明，该方法显著优于仅使用SFT的基线模型。代码已公开于https://github.com/tsunehiko/rlgdg。
原文摘要 · Abstract (English)
Game Description Generation (GDG) is the task of generating a game description written in a Game Description Language (GDL) from natural language text. Previous studies have explored generation methods leveraging the contextual understanding capabilities of Large Language Models (LLMs); however, accurately reproducing the game features of the game descriptions remains a challenge. In this paper, we propose reinforcement learning-based fine-tuning of LLMs for GDG (RLGDG). Our training method simultaneously improves grammatical correctness and fidelity to game concepts by introducing both grammar rewards and concept rewards. Furthermore, we adopt a two-stage training strategy where Reinforcement Learning (RL) is applied following Supervised Fine-Tuning (SFT). Experimental results demonstrate that our proposed method significantly outperforms baseline methods using SFT alone. Our code is available at https://github.com/tsunehiko/rlgdg
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。