arXiv:2608.15509cs.ROcs.FL2026-08中稿 · IEEE Transactions …

用时序逻辑构建通用任务表示,提升强化学习泛化与效率

Temporal Logic Guided Universal Task Representations for Reinforcement Learning

论文配图:Temporal Logic Guided Universal Task Representations for Reinforcement Learning
图 1 · 摘自论文原文
  • 基于LTL公式的任务表示架构,捕捉任务语义与关系
  • 更新机制将编码器视为策略,加速收敛20%以上
  • 理论保障行为等价性与鲁棒性,适合复杂多任务场景

任务引导的智能体在多种复杂任务中表现优异,但现有任务表示方法多依赖特定场景,难以跨域泛化,且通常需依赖强化学习控制器的梯度信号更新权重,影响表示质量与学习效率。为此,我们提出LOTUS——一种受时序逻辑启发的通用任务表示框架,可无缝集成于任意强化学习算法以提升性能。该框架设计了新型任务表示结构,能从LTL公式中建模关系并提取任务语义;引入更有效的更新机制,将LTL编码器视为策略,增强表示能力;并通过双仿真度量提供理论保证,涵盖行为等价性、最优性保真度和轨迹鲁棒性。实验表明,LOTUS在学习效率、泛化能力与表示质量上均优于多数现有方法:单任务下收敛速度提升超20%,未见操作任务成功率提高15%-45%,复杂多任务环境中子目标深度或合取增加时泛化性能提升超25%。代码、视频及附录见https://lotus-website.github.io/。

原文摘要 · Abstract (English)

Task guided agents demonstrate strong performance in a wide range of complex tasks. However, most existing task representation algorithms are tailored to specific contexts and struggle to generalize across diverse scenarios. Moreover, they typically depend on gradient signals from reinforcement learning controllers to update their weights, which can degrade both representation quality and learning efficiency. To overcome these limitations, we propose LOTUS, a temporal logic inspired universal task representation framework that can be seamlessly integrated into any RL algorithm to enhance agent performance across diverse task settings. Specifically, we design a novel task representation architecture capable of modeling relationships and extracting task semantics from LTL formulas. We further introduce a more effective update mechanism that treats the LTL encoder as a policy, thereby improving representation capacity. To enhance stability and robustness, LOTUS leverages the bisimulation metric, which provides theoretical guarantees for LTL representation, including behavioral equivalence, optimality fidelity, and trajectory robustness. Experimental results show that LOTUS outperforms most existing methods in learning efficiency, generalization capability, and representation quality. Specifically, LOTUS accelerates convergence over 20% in single-task scenarios, achieves a 15%-45% higher success rate in unseen manipulation tasks, and improves generalization performance over 25% in complex multi-task environments with increased sub-goal depth or conjunctions. The corresponding code, videos, and appendix are available at: https://lotus-website.github.io/.

强化学习时序逻辑任务表示泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。