提出强化学习框架的通用架构，统一设计标准。
A Reference Architecture of Reinforcement Learning Frameworks
- 基于18个主流框架分析，提炼通用组件与关系
- 构建可复用的参考架构，支持对比与集成
- 适合框架开发者与系统设计者参考
强化学习应用的爆发催生了多种支持技术，如强化学习框架。然而，这些框架的架构模式在实现中不一致，且缺乏统一的参考架构（RA）作为比较、评估和集成的基础。为填补这一空白，本文提出一种强化学习框架的参考架构。通过扎根理论方法，分析了18个实际使用的强化学习框架，识别出重复出现的架构组件及其相互关系，并将其编码为参考架构。为验证该架构的有效性，我们重构了典型的强化学习模式。最后，识别出若干架构趋势，如常用组件，并指出了改进强化学习框架的可行路径。
原文摘要 · Abstract (English)
The surge in reinforcement learning (RL) applications gave rise to diverse supporting technology, such as RL frameworks. However, the architectural patterns of these frameworks are inconsistent across implementations and there exists no reference architecture (RA) to form a common basis of comparison, evaluation, and integration. To address this gap, we propose an RA of RL frameworks. Through a grounded theory approach, we analyze 18 state-of-the-practice RL frameworks and, by that, we identify recurring architectural components and their relationships, and codify them in an RA. To demonstrate our RA, we reconstruct characteristic RL patterns. Finally, we identify architectural trends, e.g., commonly used components, and outline paths to improving RL frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。