arXiv:2412.00293cs.ROcs.AI2024-12

用序列模型构建自适应规划器,让机器人在未知环境中自主完成多任务。

Adaptformer: Sequence models as adaptive iterative planners

  • 基于能量函数的启发式策略,动态生成短程子目标来引导决策。
  • 在多目标迷宫任务中比现有方法提升25%成功率,可处理未见任务。
  • 适合需要长时序规划和泛化能力的机器人任务场景。

尽管基于学习的行为规划在自主系统中取得进展,但多任务任务中的决策仍具挑战性。例如,机器人需在未知环境中探索、定位目标并导航至目标,即使途中存在障碍。这类问题难解的原因在于:a) 奖励稀疏,仅当所有任务完成后才提供奖励;b) 代理需执行训练数据中未涵盖的运行时任务,如仅在所有门都打开的环境中进行示范。因此,当前最先进的方法仅限于训练数据中充分覆盖的任务且可在短规划周期内解决的任务。为克服这些限制,我们提出Adaptformer,一种随机且自适应的规划器,利用序列模型实现高效探索与利用。该框架通过学习一个需在一系列高层决策中最小化的基于能量的启发式函数。为生成长时程任务的成功动作序列,Adaptformer旨在达成更短的子目标,这些子目标通过内在的子目标课程生成。通过这两个关键组件,Adaptformer实现了对分布外任务和环境的泛化,即训练数据之外的使命。在多个仿真环境中的实证结果表明该方法的有效性。值得注意的是,Adaptformer在多目标迷宫可达性任务中比最先进方法最高提升25%,并成功适应了最先进方法无法完成的多任务使命,仅依赖单目标到达任务的示范。

原文摘要 · Abstract (English)

Despite recent advances in learning-based behavioral planning for autonomous systems, decision-making in multi-task missions remains a challenging problem. For instance, a mission might require a robot to explore an unknown environment, locate the goals, and navigate to them, even if there are obstacles along the way. Such problems are difficult to solve due to: a) sparse rewards, meaning a reward signal is available only once all the tasks in a mission have been satisfied, and b) the agent having to perform tasks at run-time that are not covered in the training data, e.g., demonstrations only from an environment where all doors were unlocked. Consequently, state-of-the-art decision-making methods in such settings are limited to missions where the required tasks are well-represented in the training demonstrations and can be solved within a short planning horizon. To overcome these limitations, we propose Adaptformer, a stochastic and adaptive planner that utilizes sequence models for sample-efficient exploration and exploitation. This framework relies on learning an energy-based heuristic, which needs to be minimized over a sequence of high-level decisions. To generate successful action sequences for long-horizon missions, Adaptformer aims to achieve shorter sub-goals, which are proposed through an intrinsic sub-goal curriculum. Through these two key components, Adaptformer allows for generalization to out-of-distribution tasks and environments, i.e., missions that were not a part of the training data. Empirical results in multiple simulation environments demonstrate the effectiveness of our method. Notably, Adaptformer not only outperforms the state-of-the-art method by up to 25% in multi-goal maze reachability tasks but also successfully adapts to multi-task missions that the state-of-the-art method could not complete, leveraging demonstrations from single-goal-reaching tasks.

强化学习任务规划机器人序列建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。