arXiv:2603.12942cs.RO2026-03被引 20

让机器人模型学会记忆,提升复杂任务表现

ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries

  • 用双层循环查询实现短时与长时记忆
  • 在多个任务上显著超越无记忆模型
  • 适合需要长期上下文的机器人控制场景

用于闭环机器人控制的视觉-语言-动作(VLA)模型通常基于马尔可夫假设,难以处理依赖历史信息的任务。为引入记忆能力,现有方法或从记忆库中检索(易受干扰),或扩展帧窗口(固定时长限制长期记忆)。本文提出ReMem-VLA,一种具备双重可学习查询的递归记忆VLA:帧级查询用于跨连续帧传递信息以支持短时记忆,块级查询用于跨时间片段携带上下文以实现长时记忆。这些查询端到端训练,持续聚合并保持相关上下文,不增加额外训练或推理开销。此外,通过引入过去观测预测作为辅助目标增强视觉记忆。在多维度的记忆相关仿真与真实机器人实验中,ReMem-VLA展现出强大的空间、序列、情景、时间及视觉记忆能力。其在记忆依赖任务上的表现显著优于无记忆基线π0.5和OpenVLA-OFT,并大幅超越MemoryVLA。

原文摘要 · Abstract (English)

Vision-language-action (VLA) models for closed-loop robot control are typically cast under the Markov assumption, making them prone to errors on tasks requiring historical context. To incorporate memory, existing VLAs either retrieve from a memory bank, which can be misled by distractors, or extend the frame window, whose fixed horizon still limits long-term retention. In this paper, we introduce ReMem-VLA, a Recurrent Memory VLA model equipped with two sets of learnable queries: frame-level recurrent memory queries for propagating information across consecutive frames to support short-term memory, and chunk-level recurrent memory queries for carrying context across temporal chunks for long-term memory. These queries are trained end-to-end to aggregate and maintain relevant context over time, implicitly guiding the model's decisions without additional training or inference cost. Furthermore, to enhance visual memory, we introduce Past Observation Prediction as an auxiliary training objective. Through extensive memory-centric simulation and real-world robot experiments, we demonstrate that ReMem-VLA exhibits strong memory capabilities across multiple dimensions, including spatial, sequential, episodic, temporal, and visual memory. ReMem-VLA significantly outperforms memory-free VLA baselines $π$0.5 and OpenVLA-OFT and surpasses MemoryVLA on memory-dependent tasks by a large margin.

机器人控制记忆机制视觉语言动作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。