提出带工作与情景记忆的非马尔可夫视觉运动策略,提升机器人长期记忆任务表现。
VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory
- 用滑动窗口和变压器压缩器构建工作记忆与情景记忆
- 在MIKASA上比基线提升20%以上,MoMaRT平均提升5%
- 适合需要长时记忆的机器人操作任务,兼顾实时性与内存效率
从人类示范中学习的模仿学习在机器人控制中取得显著进展,但多数视觉运动策略仅依赖单步观测或短历史,难以应对需长期记忆的非马尔可夫任务。单纯扩大上下文窗口会带来巨大计算与内存开销,并导致对虚假相关性的过拟合,引发分布外失效且违反机器人系统的实时性要求。相比之下,人类能将重要经历压缩为长期记忆并终身利用。本文提出VPWEM,一种具备工作记忆与情景记忆的非马尔可夫视觉运动策略。它保留最近观察嵌入的滑动窗口作为短期工作记忆,并引入基于Transformer的上下文记忆压缩器,递归地将窗口外的观测转化为固定数量的情景记忆嵌入。该压缩器通过自注意力聚合过往摘要嵌入,通过交叉注意力关联历史观测,与策略联合训练。我们在扩散策略上实现VPWEM,以近乎恒定的每步内存与计算量融合短期与全周期信息生成动作。实验表明,VPWEM在记忆密集型任务集MIKASA上优于最先进基线(包括扩散策略与视觉-语言-动作模型)超过20%,在移动操作基准MoMaRT上平均提升5%。代码已开源。
原文摘要 · Abstract (English)
Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory. Simply enlarging the context window incurs substantial computational and memory costs and encourages overfitting to spurious correlations, leading to catastrophic failures under distribution shift and violating real-time constraints in robotic systems. By contrast, humans can compress important past experiences into long-term memories and exploit them to solve tasks throughout their lifetime. In this paper, we propose VPWEM, a non-Markovian visuomotor policy equipped with working and episodic memories. VPWEM retains a sliding window of recent observation embeddings as short-term working memory, and introduces a Transformer-based contextual memory compressor that recursively converts out-of-window observations into a fixed number of episodic memory embeddings. The compressor uses self-attention over a cache of past summary embeddings and cross-attention over a cache of historical observations, and is trained jointly with the policy. We instantiate VPWEM on diffusion policies to exploit both short-term and episode-wide information for action generation with nearly constant memory and computation per step. Experiments demonstrate that VPWEM outperforms state-of-the-art baselines including diffusion policies and vision-language-action (VLA) models by more than 20\% on the memory-intensive manipulation tasks in MIKASA and achieves an average 5\% improvement on the mobile manipulation benchmark MoMaRT. Code is available at https://github.com/HarryLui98/code_vpwem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。