用分阶段探索与状态感知记忆提升机器人长程操作成功率。
Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory

- 将长任务拆解为短周期子任务,逐个探索并存入记忆
- 在RoboMemArena上任务成功率提升11.6%,累计成功率达14.9%新高
- 适合需要稳定执行多步复杂操作的机器人研发场景
长时序机器人操作需将多个接触密集型技能串联成多阶段任务。尽管视觉-语言-动作(VLA)模型已掌握单个技能,但整体链条仍易失败:错误累积超出纠错能力,且前一子任务无声影响后续。现有方法冻结VLA,由大语言模型(LLM)代理规划语言指令,以解析基元在自由空间移动,仅在接触密集段调用VLA,并将适应性写入语言记忆。但该方法在长时序任务中面临双重挑战:(1) 测试时全任务探索成本随阶段数呈指数增长(若每阶段需T次试验,K阶段任务需约T^K次),且失败原因难以定位;(2) 缺乏对状态转换的建模:VLA原语仅有退出条件而无进入条件,导致子任务成功形式可能不被下一阶段使用。本文提出BATON。针对(1),将子任务作为探索单位,以低成本短周期方式分别探索并存储解决方案,长时序轨迹由此组合而成,成本变为加性(T×K),失败可精准归因于单一阶段。针对(2),引入状态感知记忆:子任务内,验证代理控制调用时机,仅当腕部视角确认场景就绪才触发VLA;跨子任务间,交接转换恢复前一阶段残留扰动的状态,前瞻转换选择后继可继承结果的策略。全程不更新参数。在长时序基准RoboMemArena上,BATON相较当前最优方法,任务成功率提升11.6%,累计成功率提升14.9%。
原文摘要 · Abstract (English)
Long-horizon robot manipulation chains many contact-rich skills into one multi-stage task. Vision-language-action (VLA) models increasingly master the individual skills, yet the chain still fails: errors compound beyond the policy's ability to correct, and one subtask silently constrains the next. A promising recipe freezes the VLA and puts an LLM agent in charge: it plans in language, moves in free space with analytic primitives, invokes the VLA only for contact-rich segments, and writes adaptation into language memory. Applied to long horizons, it breaks twice. (1) Competence comes from whole-task exploration at test time, whose cost is multiplicative in stages: if one stage needs T episodes, a K-stage task needs about T^K, and a failure does not reveal which stage caused it. (2) It has no representation of transitions: the VLA primitive carries an exit but no entry condition, so a subtask can succeed in a form its successor cannot use. We present BATON. Against (1), BATON makes the subtask the unit of exploration: each is explored in the cheap short-horizon regime and its solution stored in memory; a long-horizon trajectory is then composed from these solutions rather than discovered whole. Cost becomes additive (T*K) and every failure is attributed to a single stage. Against (2), BATON equips exploration with a transition-aware memory. Within a subtask, a verifier agent governs the invocation transition: the VLA is called only after the wrist view confirms the scene is ready. Across subtasks, a handoff transition restores an entry state disturbed by the predecessor's residue, and a lookahead transition selects the strategy whose outcome the successor can inherit. No parameters are updated. On the long-horizon benchmark RoboMemArena, BATON improves task success by 11.6% and cumulative success by 14.9% over the SoTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。