用图神经网络提升控制器合成的探索效率
Graph Contextual Reinforcement Learning for Efficient Directed Controller Synthesis
- 用图结构编码控制器搜索历史，捕捉非当前状态的上下文信息
- 在五个基准测试中，四域表现优于现有方法，学习更快更泛化
- 适合需要高效自动控制器生成的系统验证与安全关键场景
控制器合成是一种通过形式化方法自动生成满足特定性质的带标签转移系统（LTS）控制器的技术。然而，合成过程的效率高度依赖于探索策略。现有策略通常基于固定规则或仅考虑有限当前特征的强化学习方法。为解决这一局限，本文提出GCRL，通过引入图神经网络（GNN）增强基于强化学习的方法。GCRL将LTS探索的历史编码为图结构，从而捕捉更广泛的、非依赖当前状态的上下文信息。在与前沿方法的对比实验中，GCRL在五个基准领域中的四个表现出更高的学习效率和更强的泛化能力，仅在一个具有高对称性且交互严格局部的领域表现不佳。
原文摘要 · Abstract (English)
Controller synthesis is a formal method approach for automatically generating Labeled Transition System (LTS) controllers that satisfy specified properties. The efficiency of the synthesis process, however, is critically dependent on exploration policies. These policies often rely on fixed rules or strategies learned through reinforcement learning (RL) that consider only a limited set of current features. To address this limitation, this paper introduces GCRL, an approach that enhances RL-based methods by integrating Graph Neural Networks (GNNs). GCRL encodes the history of LTS exploration into a graph structure, allowing it to capture a broader, non-current-based context. In a comparative experiment against state-of-the-art methods, GCRL exhibited superior learning efficiency and generalization across four out of five benchmark domains, except one particular domain characterized by high symmetry and strictly local interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。