用大模型自动设计赛车游戏奖励函数，生成媲美冠军级智能体的新玩法。
Automated Reward Design for Gran Turismo
- 基于文本指令，用大模型生成多种奖励函数组合。
- 结合视觉语言模型与人工反馈，筛选出性能接近GT Sophy的智能体。
- 适合想快速构建复杂行为的强化学习研究者或游戏开发者。
在设计强化学习（RL）智能体时，设计者通过定义奖励函数来传达期望的行为——即对智能体动作给予数值反馈。然而，在自动驾驶赛车等复杂环境中，将期望行为映射为有效奖励函数极具挑战。本文展示如何利用当前基础模型，在仅提供文本指令的情况下，高效搜索奖励函数空间，为《Gran Turismo 7》生成表现优异的RL智能体。通过结合大语言模型生成奖励、视觉语言模型进行偏好评估以及人工反馈优化，我们的系统不仅生成了性能媲美冠军级智能体GT Sophy的赛车代理，还探索出新颖行为模式，为真实世界应用中的自动化奖励设计提供了可行路径。
原文摘要 · Abstract (English)
When designing reinforcement learning (RL) agents, a designer communicates the desired agent behavior through the definition of reward functions - numerical feedback given to the agent as reward or punishment for its actions. However, mapping desired behaviors to reward functions can be a difficult process, especially in complex environments such as autonomous racing. In this paper, we demonstrate how current foundation models can effectively search over a space of reward functions to produce desirable RL agents for the Gran Turismo 7 racing game, given only text-based instructions. Through a combination of LLM-based reward generation, VLM preference-based evaluation, and human feedback we demonstrate how our system can be used to produce racing agents competitive with GT Sophy, a champion-level RL racing agent, as well as generate novel behaviors, paving the way for practical automated reward design in real world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。