用时序逻辑指导机器人任务，提升长程操作的效率与可解释性。
Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation
- 分三层决策：高层用时序逻辑规划路径，中层选行为，底层调参数。
- 在4个复杂操作任务上实现更快收敛，避免任务失败。
- 适合需要高可解释性的机器人控制场景，如医疗或工业协作。
基于强化学习的方法在机器人学习中日益受到关注，但通常在探索阶段采样效率低，尤其在长时程操作任务中，且普遍忽略任务层面的语义信息，导致收敛延迟甚至任务失败。为此，我们提出一种时序逻辑引导的混合策略框架（HyTL），采用三级决策结构以提升智能体性能。具体地，通过线性时序逻辑（LTL）编码任务规范，增强性能并提供可解释性；设计一个基于LTL编码任务层反馈的航点规划模块作为高层策略，提升探索效率；中层策略选择执行的行为原语，底层策略指定与环境交互的相应参数。我们在四个具有挑战性的操作任务上评估了HyTL，验证了其有效性和可解释性。项目代码与资料见：https://sites.google.com/view/hytl-0257/。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To tackle these challenges, we propose a Temporal-Logic-guided Hybrid policy framework (HyTL) which leverages three-level decision layers to improve the agent's performance. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy selects which behavior primitives to execute, and the low-level policy specifies the corresponding parameters to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. Our project is available at: https://sites.google.com/view/hytl-0257/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。