让冻结的视觉语言动作模型具备闭环执行能力,靠物理证据验证进展。
AGM: Achievement-Grounded Memory for Closed-Loop Agents with Frozen VLA Policies

- 用子目标序列和进度指针构建任务记忆,仅在物理验证后更新
- 在RoboMME计数任务上表现优于最强基线,平均提升12.3点
- 轻量级设计,无需解冻模型或增加推理开销,适合真实机器人部署
冻结的视觉语言动作(VLA)策略具备广泛操作能力,但以开环方式执行动作块,无法追踪任务进展,难以判断是否继续、重试或终止。外部记忆是自然解决方案,但若将尝试动作视为已完成进展,会将局部执行错误转为持久的任务状态错误。本文提出成就接地记忆(AGM),一种针对冻结VLA策略的轻量级闭环框架,将任务表示为带进度指针的子目标序列,仅在当前子目标通过物理证据验证后才推进记忆。本体感知交互线索决定何时验证,而来自冻结基础模型的统一点追踪与语言条件跨视图对比,通过一个仅2.43万参数的验证头实现。AGM将开环执行转化为执行-验证-进展的闭环,保持策略冻结且无需测试时大模型推理。在RoboMME Counting基准上,AGM在PickXTimes任务上达到96.7%成功率,在BinFill任务上达89.4%,平均超越最强记忆增强基线12.3点,且在真实机器人上也取得显著提升。可靠具身记忆更依赖于严谨的状态更新,而非记忆容量。
原文摘要 · Abstract (English)
Frozen vision-language-action (VLA) policies offer broad manipulation skills but execute open-loop action chunks without tracking task progress, so the agent cannot reliably decide whether to continue, retry, or terminate. External memory is a natural remedy, yet it can be harmful when attempted actions are treated as completed progress, turning local execution errors into persistent task-state errors. We propose Achievement-Grounded Memory (AGM), a lightweight closed-loop framework for frozen VLA policies that represents a task as a subgoal sequence with a progress pointer and advances this memory only after the current subgoal is verified by physical evidence. Proprioceptive interaction cues decide when to verify, while coherent point tracking and language-conditioned cross-view comparison, sourced from frozen foundation models through a single 2.43M-parameter verification head, decide what was achieved. AGM thereby converts open-loop execution into a closed loop of execution, verification, and progress, keeping the policy frozen without test-time large-model inference. On the RoboMME Counting benchmark, AGM reaches on PickXTimes and on BinFill, surpassing the strongest memory-augmented baseline by points on average, and the framework yields equally decisive gains on a physical robot. Reliable embodied memory thus depends more on disciplined state updates than on memory capacity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。