用逻辑公式结构化表示任务，让智能体零样本执行新指令
Zero-Shot Instruction Following in RL via Structured LTL Representations
- 用有限状态机生成布尔公式序列，作为任务条件输入
- 在复杂环境中实现零样本泛化，优于现有方法
- 适合需要灵活应对新任务的强化学习应用
我们研究多任务强化学习中的指令跟随问题，即智能体需在未见过的任务上零样本执行。近年来，线性时序逻辑（LTL）被用于描述具有时序结构的复杂任务。现有方法虽能训练通用策略，但难以有效捕捉LTL规范中蕴含的丰富逻辑与时序结构。为此，本文提出一种新方法：将任务条件建模为由任务有限自动机生成的布尔公式序列，并设计分层神经网络编码其逻辑结构，引入注意力机制使策略能推理未来子目标。在多种复杂环境中的实验表明，该方法具备强大的泛化能力与优越性能。
原文摘要 · Abstract (English)
We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training. In this setting, linear temporal logic (LTL) has recently been adopted as a powerful framework for specifying structured, temporally extended tasks. While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in LTL specifications. In this work, we address these concerns with a novel approach to learn structured task representations that facilitate training and generalisation. Our method conditions the policy on sequences of Boolean formulae constructed from a finite automaton of the task. We propose a hierarchical neural architecture to encode the logical structure of these formulae, and introduce an attention mechanism that enables the policy to reason about future subgoals. Experiments in a variety of complex environments demonstrate the strong generalisation capabilities and superior performance of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。