用模型驱动方法自动生成相似但不同的强化学习训练环境。
A Model-Driven Approach for Developing Families of Reinforcement Learning Environments
- 结合遗传算法与模型变换，自动构造环境变体族。
- 在野火防控场景中验证，支持课程学习的渐进式训练。
- 适合需要大量环境变体的RL研究者和系统开发者。
虚拟训练环境是强化学习（RL）智能体学习、适应并展现有意义行为的软件密集型系统。它们为在真实世界中训练智能体提供了安全且低成本的替代方案。然而，大多数现实的强化学习问题需要在多个高度相似但略有差异的环境中进行训练——即环境变体族。当前环境族的开发过程主要依赖人工，费时且易出错，难以扩展。为此，本文提出一种模型驱动的方法来开发强化学习训练环境族。我们设计了一种混合遗传算法，融合基于种群的全局搜索与启发式局部搜索，通过模型变换表达突变与约束，并由先进的模型变换引擎实现搜索过程。我们在野火防控场景和课程学习（一种依赖环境族的学习范式）中验证了该方法的有效性。
原文摘要 · Abstract (English)
Virtual training environments are software-intensive systems in which reinforcement learning (RL) agents learn, adapt, and demonstrate meaningful behavior. Virtual training environments offer a safe and cost-efficient alternative to training agents in real-world settings. However, to converge, most realistic RL problems require training in multiple, mostly similar but slightly different environments - i.e., families of environment variants. The typical development process of environment families is a labor-intensive and error-prone manual endeavor that does not scale well. To alleviate these issues, in this paper, we propose a model-driven approach for developing families of RL training environments. To obtain the family of environments, we develop an approach and prototype tool. In our approach, a hybrid genetic algorithm - a combination of population-based global search and heuristic local search - generates environment families. Mutations and constraints are expressed as model transformations and are operationalized into a search process by a state-of-the-art model transformation engine. We demonstrate the soundness of our approach in a wildfire mitigation scenario and curriculum learning - a particular learning paradigm that relies on environment families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。