用视觉语言模型自动标注游戏奖励，让智能体学会按需行动。
Training a Conditioned Video Game Agent on a VLM Annotated Dataset
- 用VLM解析游戏画面提取人类定义的奖励信号
- 离线强化学习训练出能响应目标回报的条件化智能体
- 解决奖励设计难、稀疏和不可解释等问题，适合游戏AI研究者
强化学习在策略学习中虽强大，但应用复杂。以视频游戏为例，需访问游戏引擎获取奖励，且奖励的设计与加权通常依赖繁琐的试错。此外，奖励常稀疏，其对策略的影响也难以理解。为此，我们提出使用视觉语言模型（VLM）对游戏数据集进行标注，引导其提取人类定义的奖励。实验表明，基于此标注数据，可采用离线强化学习训练出能响应特定回报的条件化智能体，并讨论了早期实验中遇到的挑战与局限。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) is a powerful but far from easy-to-use technique for policy learning. In the specific case of video games, access to the game engine is required to get rewards for training (e.g. to collect rewards from the environment). Furthermore, the proper identification and weighting of the rewards generally requires a difficult trial-and-error approach. Lastly, rewards are often sparse and understanding how they eventually affect the learned policy is a non-trivial exercise. To ease these issues we propose annotating a video game dataset with Vision Language Models (VLMs) instructed to extract human defined rewards. We show that offline RL can then be used to train a conditioned agent that responds accordingly to the desired returns and we discuss the difficulties and limitations that emerged in our early experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。