让机器人像人一样记忆关键经验，实现长时间复杂操作
MemER: Scaling Up Memory for Robot Control via Experience Retrieval
- 分层策略：高层选关键帧，低层结合新旧帧生成指令
- 在3个真实任务中表现超越现有方法，支持数分钟级记忆
- 兼容现有视觉语言模型，仅需少量语言标注即可训练
人类常依赖记忆完成任务，但多数机器人策略缺乏此能力。直接使用长序列观察历史计算成本高且易受分布偏移影响，而盲目采样则会引入无关或冗余信息。本文提出分层策略框架，高层策略学习从经验中选择并跟踪相关关键帧，结合最新帧为低层策略生成文本指令。该设计兼容现有视觉-语言-动作（VLA）模型，可高效处理长时程依赖。实验中，以Qwen2.5-VL-7B-Instruct和π_{0.5}分别作为高低层策略，通过少量语言标注的示范数据微调。所提方法MemER在三个需数分钟记忆的真实世界长时程机器人操作任务中优于先前方法。视频与代码见https://jen-pan.github.io/memer/
原文摘要 · Abstract (English)
Humans routinely rely on memory to perform tasks, yet most robot policies lack this capability; our goal is to endow robot policies with the same ability. Naively conditioning on long observation histories is computationally expensive and brittle under covariate shift, while indiscriminate subsampling of history leads to irrelevant or redundant information. We propose a hierarchical policy framework, where the high-level policy is trained to select and track previous relevant keyframes from its experience. The high-level policy uses selected keyframes and the most recent frames when generating text instructions for a low-level policy to execute. This design is compatible with existing vision-language-action (VLA) models and enables the system to efficiently reason over long-horizon dependencies. In our experiments, we finetune Qwen2.5-VL-7B-Instruct and $π_{0.5}$ as the high-level and low-level policies respectively, using demonstrations supplemented with minimal language annotations. Our approach, MemER, outperforms prior methods on three real-world long-horizon robotic manipulation tasks that require minutes of memory. Videos and code can be found at https://jen-pan.github.io/memer/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。