用协程语法简化强化学习环境开发，提升效率与可维护性。
Rulebook: bringing co-routines to reinforcement learning environments
- 采用协程机制自动编译生成状态机，无需手动管理状态
- 相比传统方式减少同步开销，开发成本降低且环境更复杂
- 适合希望快速构建复杂仿真环境的RL研究者
强化学习算法依赖外部系统学习，需使用接口简单的数字环境（如模拟器），这严重限制了环境实现。当前环境通常以独立进程或状态机形式实现，前者导致同步与通信开销，后者造成编程结构混乱。本文提出一种新的领域专用、基于协程的编译语言Rulebook，可自动生成与机器学习算法交互所需的状态机，无性能损耗。Rulebook使用户无需关注ML组件的具体接口要求，通过解耦程序执行模型与语法表达，避免手动状态管理，从而以更低开发成本构建更大更复杂的环境。
原文摘要 · Abstract (English)
Reinforcement learning (RL) algorithms, due to their reliance on external systems to learn from, require digital environments (e.g., simulators) with very simple interfaces, which in turn constrain significantly the implementation of such environments. In particular, these environments are implemented either as separate processes or as state machines, leading to synchronization and communication overheads in the first case, and to unstructured programming in the second. We propose a new domain-specific, co-routine-based, compiled language, called Rulebook, designed to automatically generate the state machine required to interact with machine learning (ML) algorithms and similar applications, with no performance overhead. Rulebook allows users to express programs without needing to be aware of the specific interface required by the ML components. By decoupling the execution model of the program from the syntactical encoding of the program, and thus without the need for manual state management, Rulebook allows to create larger and more sophisticated environments at a lower development cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。