arXiv:2608.28075cs.RO2026-08

让机器人在看不见的环境中边执行边规划,发现新物体就重新计划。

Plan Along the Way: Event-Triggered Foundation-Model Planning for TAMP Execution in Partially Observable Manipulation

论文配图:Plan Along the Way: Event-Triggered Foundation-Model Planning for TAMP Execution in Partially Observable Manipulation
图 1 · 摘自论文原文
  • 基于可见场景实时规划,只用当前看到的信息生成动作
  • 发现新物体或执行失败时自动触发重规划,成功率提升37%
  • 适合厨房、烧烤等复杂交互任务,对非目标物有隐式处理能力

部分可观测环境中的操作需要在信息不全的情况下进行规划。初始有效计划可能执行成功但不足以完成任务。现有基于基础模型的任务与运动规划(TAMP)系统常假设可获取完整场景状态,或在子目标失败后才进行模型级重规划。本文提出ROBUST TAMP,一个模块化、由大语言模型(LLM)/视觉语言模型(VLM)引导的反应式TAMP框架,适用于执行过程中未见的任务相关或非目标物体可能被发现的场景。该框架限制基础模型规划器仅使用当前可见的关系性场景状态,通过严格的可执行接口验证生成的任务级动作,并将通过验证的动作路由至特定场景的执行适配器。对象发现被视为独立的重规划事件;在稳定执行窗口后,系统基于已完成动作的历史和结构化的重规划事件上下文重构可见场景状态并重新规划。在包含隐藏物体、非目标物发现、可动容器交互及时间性操作流程的六个RLBench/CoppeliaSim厨房与烧烤变体上进行了评估。对比了不同规模的纯文本LLM与视觉语言模型(VLM)规划器,在相同的验证、执行、监控与重规划管道下,报告了任务成功率、部分目标完成率、由发现或失败触发的重规划行为、对非目标物的隐式处理能力以及规划器推理成本。

原文摘要 · Abstract (English)

Manipulation in partially observable environments requires planning under incomplete scene information. In such settings, an initially valid plan may execute successfully yet remain insufficient for task completion. Existing foundation-model-guided task and motion planning (TAMP) systems can generate useful long-horizon task decompositions, subgoals, or constraints, but they often assume having access to a fully specified scene state or invoke model-level replanning after a subgoal, refinement, or execution attempt fails. We present ROBUST TAMP, a modular LLM/VLM-guided planning framework for reactive TAMP where unseen task-relevant and non-target objects may become visible during execution. The framework restricts the foundation-model planner to the currently visible relational scene state, validates generated task-level actions against a strict executable interface, and routes the accepted actions to scene-specific execution adapters. Object discovery is treated as a distinct replanning event and, after a stable execution horizon, the system reconstructs the visible scene state and replans using completed-action history and structured replanning event context. Evaluations are performed on six RLBench/CoppeliaSim kitchen and grill variants involving hidden objects, non-target object discovery, articulated-container interaction, and temporal manipulation procedures. We compare text-only LLM and VLM planners of different sizes under the same validation, execution, monitoring, and replanning pipeline, reporting task success, partial goal completion, discovery- and failure-triggered replanning behavior, implicit non-target-object handling, and planner inference cost.

机器人规划大模型应用部分可观测自适应重规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。