用大模型零样本玩通Frogger，还能提升传统强化学习效率
Frog Soup: Zero-Shot, In-Context, and Sample-Efficient Frogger Agents
- 用具备域外强化学习能力的大模型直接玩Frogger，无需训练
- 上下文学习和推理投入越多，模型表现越好，最高通关率超70%
- 用大模型生成示范数据，让传统RL方法少训练10倍样本
强化学习研究的核心目标之一是开发能快速适应新任务的通用智能体。尽管强化学习游戏智能体已掌握多种Atari游戏，但对每款游戏仍需耗时耗力地训练。本文展示，最新具备域外强化学习微调能力的大语言模型可在零样本设定下成功游玩具有挑战性的Atari游戏Frogger。我们进一步研究了上下文学习和推理投入对模型性能的影响。最后，我们提出一种利用大模型生成示范数据来引导传统强化学习的方法，显著提升了其性能与样本效率。实现代码已开源：https://github.com/AlienKevin/frogger。
原文摘要 · Abstract (English)
One of the primary aspirations in reinforcement learning research is developing general-purpose agents capable of rapidly adapting to and mastering novel tasks. While RL gaming agents have mastered many Atari games, they remain slow and costly to train for each game. In this work, we demonstrate that latest reasoning LLMs with out-of-domain RL post-training can play a challenging Atari game called Frogger under a zero-shot setting. We then investigate the effect of in-context learning and the amount of reasoning effort on LLM performance. Lastly, we demonstrate a way to bootstrap traditional RL method with LLM demonstrations, which significantly improves their performance and sample efficiency. Our implementation is open sourced at https://github.com/AlienKevin/frogger.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。