用图神经网络增强LTL指令理解，让智能体零样本执行复杂任务
Zero-Shot Instruction Following in RL via Structured LTL Representations
- 用布尔公式序列替代传统LTL，通过图神经网络编码任务结构
- 在棋类环境中实现零样本多任务执行，支持多个事件同时发生
- 适合需要灵活应对复杂逻辑指令的强化学习场景
线性时序逻辑（LTL）是为强化学习（RL）智能体指定复杂结构化任务的有力框架。近期工作表明，将LTL指令解释为有限自动机（可视为监控任务进展的高层程序），能训练出一个通用策略，在测试时执行任意指令。然而，现有方法在多个基本命题（原子命题）可同时为真且相互作用复杂的环境中表现不佳。本文提出一种新方法，通过条件化策略于简单布尔公式的序列（与自动机转移直接对应），并利用图神经网络（GNN）编码以生成结构化任务表示，从而解决该问题。实验在复杂的棋类环境中验证了该方法的优势，展示了其在零样本条件下执行多样化指令的能力。
原文摘要 · Abstract (English)
Linear temporal logic (LTL) is a compelling framework for specifying complex, structured tasks for reinforcement learning (RL) agents. Recent work has shown that interpreting LTL instructions as finite automata, which can be seen as high-level programs monitoring task progress, enables learning a single generalist policy capable of executing arbitrary instructions at test time. However, existing approaches fall short in environments where multiple high-level events (i.e., atomic propositions) can be true at the same time and potentially interact in complicated ways. In this work, we propose a novel approach to learning a multi-task policy for following arbitrary LTL instructions that addresses this shortcoming. Our method conditions the policy on sequences of simple Boolean formulae, which directly align with transitions in the automaton, and are encoded via a graph neural network (GNN) to yield structured task representations. Experiments in a complex chess-based environment demonstrate the advantages of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。