让强化学习课程可移植,轻松适配各类训练框架。
Syllabus: Portable Curricula for Reinforcement Learning Agents
- 提供通用API,支持多种自动课程学习方法
- 在NetHack和Neural MMO中首次实现自动课程学习
- 代码轻量易集成,适合跨框架研究者使用
课程学习是众多强化学习成功案例中的关键因素,但目前仍是小众领域,未被主流RL库直接支持。此类方法能提升智能体能力与泛化性,却常需复杂代码修改。我们提出Syllabus——一个可移植的课程学习库,提供通用API、模块化实现的主流自动课程学习方法,以及可与几乎任何RL库的异步训练代码无缝集成的基础设施。Syllabus采用极简API设计核心组件,便于新算法开发与现有算法迁移至新环境。我们在多个新环境中验证了其有效性,使用不同RL库编写的智能体均成功应用。首次在最具挑战性的基准测试NetHack和Neural MMO中实现自动课程学习,并发现现有方法无法直接迁移至复杂新环境。
原文摘要 · Abstract (English)
Curriculum learning has been a quiet, yet crucial component of many high-profile successes of reinforcement learning. Despite this, it is still a niche topic that is not directly supported by any of the major reinforcement learning libraries. These methods can improve the capabilities and generalization of RL agents, but often require complex changes to training code. We introduce Syllabus, a portable curriculum learning library, as a solution to this problem. Syllabus provides a universal API for curriculum learning, modular implementations of popular automatic curriculum learning methods, and infrastructure that allows them to be easily integrated with asynchronous training code in nearly any RL library. Syllabus provides a minimal API for core curriculum learning components, making it easier to design new algorithms and adapt existing ones to new environments. We demonstrate this by evaluating the algorithms in Syllabus on several new environments, each using agents written in a different RL library. We present the first examples of automatic curriculum learning in NetHack and Neural MMO, two of the most challenging RL benchmarks, and find evidence that existing methods do not directly transfer to complex new environments. Syllabus can be found at https://github.com/RyanNavillus/Syllabus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。