用带不确定反馈的强化学习，自动优化复杂模型转换序列。
Complex Model Transformations by Reinforcement Learning with Uncertain Human Guidance
- 将用户定义的模型转换映射为强化学习动作，实现自动化序列生成。
- 即使人类指导有不确定性，也能显著提升强化学习在复杂任务中的表现。
- 适合需要人机协作的工程化建模场景，如系统修复与设计探索。
模型驱动工程常需执行复杂的模型转换（MTs），即一系列长链式转换操作，典型应用包括模型同步、自动修复和设计空间探索。手动构建此类复杂转换序列易出错且难以实现。强化学习（RL）可通过试错探索状态空间，发现有效的转换序列，但面对复杂问题时性能受限。本文提出一种基于可能不确定的人类指导的强化学习框架，用于生成复杂模型转换序列。该框架将用户自定义的转换操作映射为强化学习原语，并以程序形式执行，以寻找最优转换路径。评估表明，即便人类指导存在不确定性，也能显著改善强化学习性能，加速复杂转换序列的开发。通过权衡人类建议的确定性与及时性，本方法推动了面向人机协同的强化学习工程实践发展。
原文摘要 · Abstract (English)
Model-driven engineering problems often require complex model transformations (MTs), i.e., MTs that are chained in extensive sequences. Pertinent examples of such problems include model synchronization, automated model repair, and design space exploration. Manually developing complex MTs is an error-prone and often infeasible process. Reinforcement learning (RL) is an apt way to alleviate these issues. In RL, an autonomous agent explores the state space through trial and error to identify beneficial sequences of actions, such as MTs. However, RL methods exhibit performance issues in complex problems. In these situations, human guidance can be of high utility. In this paper, we present an approach and technical framework for developing complex MT sequences through RL, guided by potentially uncertain human advice. Our framework allows user-defined MTs to be mapped onto RL primitives, and executes them as RL programs to find optimal MT sequences. Our evaluation shows that human guidance, even if uncertain, substantially improves RL performance, and results in more efficient development of complex MTs. Through a trade-off between the certainty and timeliness of human advice, our method takes a step towards RL-driven human-in-the-loop engineering methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。