用生成模型让机器人自适应规划,能搞定训练时没见过的任务和环境。
GenPlan: Generative Sequence Models as Adaptive Planners
- 基于离散流生成模型迭代去噪,动态生成目标与动作序列。
- 在未知任务和环境中表现优于现有方法超10%。
- 适合需要灵活应对新任务的智能体规划场景。
序列模型在行为规划中已取得显著成果,依赖已有示范数据。但面对多任务使命时仍面临挑战,尤其当规划器需适应未见约束与任务(如发现目标、解锁门禁)时。问题根源在于:一、智能体无法超越单一任务的奖励函数进行适应;二、在仅于平面环境训练的情况下,难以泛化到含墙壁、锁门等新环境。因此,当前顶尖决策方法仅限于任务在训练数据中有充分覆盖且规划时间较短的情况。为此,我们提出GenPlan:一种基于离散流模型的随机性自适应规划器,支持样本高效探索与利用。该框架通过迭代去噪生成目标与动作序列,捕捉多模态动作分布,实现目标与任务发现,从而泛化至分布外的任务与环境(即训练数据中不存在的使命)。我们在多个仿真环境中验证了方法的有效性。结果表明,GenPlan在自适应规划任务上性能优于现有最优方法超过10%,且能利用单目标到达示范完成多任务适应。代码已开源:https://github.com/CL2-UWaterloo/GenPlan。
原文摘要 · Abstract (English)
Sequence models have demonstrated remarkable success in behavioral planning by leveraging previously collected demonstrations. However, solving multi-task missions remains a significant challenge, particularly when the planner must adapt to unseen constraints and tasks, such as discovering goals and unlocking doors. Such behavioral planning problems are challenging to solve due to: a) agents failing to adapt beyond the single task learned through their reward function, and b) inability to generalize to new environments, e.g., those with walls and locked doors, when trained only in planar environments. Consequently, state-of-the-art decision-making methods are limited to missions where the required tasks are well-represented in the training demonstrations and can be solved within a short (temporal) planning horizon. To address this, we propose GenPlan: a stochastic and adaptive planner that leverages discrete-flow models for generative sequence modeling, enabling sample-efficient exploration and exploitation. This framework relies on an iterative denoising procedure to generate a sequence of goals and actions. This approach captures multi-modal action distributions and facilitates goal and task discovery, thereby generalizing to out-of-distribution tasks and environments, i.e., missions not part of the training data. We demonstrate the effectiveness of our method through multiple simulation environments. Notably, GenPlan outperforms state-of-the-art methods by over 10% on adaptive planning tasks, where the agent adapts to multi-task missions while leveraging demonstrations from single-goal-reaching tasks. Our code is available at https://github.com/CL2-UWaterloo/GenPlan.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。