给视觉语言动作模型加记忆,让机器人完成复杂长任务更稳。
MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulation
- 用演示数据构建可学习的记忆提示库,记录任务各阶段信息。
- 实时匹配并融合相关记忆,使长程任务成功率提升25%。
- 无需修改原模型,轻量插件式设计,适合各类机器人系统。
预训练的视觉-语言-动作(VLA)模型在端到端机器人操作中表现出色,但因缺乏记忆且仅依赖即时感官输入,在长程任务上表现不佳。为此,我们提出记忆增强提示框架MAP-VLA,通过演示数据生成记忆提示,增强VLA模型在长程操作中的动作生成能力。MAP-VLA首先从历史演示中构建记忆库,每个记忆单元以可学习的软提示形式存储特定任务阶段信息,并通过提示调优优化。执行时,通过轨迹相似性匹配检索相关记忆,并动态融入VLA模型以增强决策。该方法作为即插即用模块运行于冻结的VLA模型之上,兼具轻量与灵活性。实验表明,MAP-VLA在仿真基准中性能提升达7.0%,在真实机器人测试中提升25.0%,优于当前最优方法。
原文摘要 · Abstract (English)
Pre-trained Vision-Language-Action (VLA) models have achieved remarkable success in improving robustness and generalization for end-to-end robotic manipulation. However, these models struggle with long-horizon tasks due to their lack of memory and reliance solely on immediate sensory inputs. To address this limitation, we propose Memory-Augmented Prompting for Vision-Language-Action model (MAP-VLA), a novel framework that empowers pre-trained VLA models with demonstration-derived memory prompts to augment action generation for long-horizon robotic manipulation tasks. To achieve this, MAP-VLA first constructs a memory library from historical demonstrations, where each memory unit captures information about a specific stage of a task. These memory units are implemented as learnable soft prompts optimized through prompt tuning. Then, during real-time task execution, MAP-VLA retrieves relevant memory through trajectory similarity matching and dynamically integrates it into the VLA model for augmented action generation. Importantly, this prompt tuning and retrieval augmentation approach operates as a plug-and-play module for a frozen VLA model, offering a lightweight and flexible solution to improve task performance. Experimental results show that MAP-VLA delivers up to 7.0% absolute performance gains in the simulation benchmark and 25.0% on real robot evaluations for long-horizon tasks, surpassing the current state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。