测试机器人在干扰下的长期记忆能力,发现现有系统表现大幅下降。
RoboMME-Interference: Benchmarking Robot Memory Under Interference
- 构建跨会话记忆测试集,模拟真实环境中的干扰
- 无干扰时感知记忆模型表现好,但干扰增多后性能急剧下滑
- 引入视觉相似性检索可有效提升抗干扰能力,适合长期部署场景
实际部署的机器人需在多个会话和任务中积累经验,常需回忆数个会话前的信息,因此长上下文记忆对现实应用至关重要。然而,当前大多数机器人记忆基准仅基于单次会话或短上下文。为评估现有系统在更长会话与更多干扰下的表现,我们提出 RoboMME-Interference,基于 RoboMME(Dai et al., 2026)构建的跨会话基准。每个查询会话通过相关历史演示加上可控数量的无关会话构成记忆历史,输入至视觉语言代理(VLA)并测量准确率。在未修改的 RoboMME 释放的 memory-augmented $π_{0.5}$ 变体上测试发现,感知记忆变体在无干扰时成功率提升,但随着无关会话累积,性能持续且显著下降。子目标变体虽最佳表现提升有限,但在干扰增加时仍能保持较高性能。在最强感知变体中加入检索步骤——仅选取与当前视觉最相似的历史片段传递给策略——可在所有干扰水平下恢复无干扰时的成功率。本研究强调长上下文记忆与抗干扰鲁棒性的关键作用,并表明当前系统在该能力上普遍不足。项目页面、视频、代码与数据见 https://robotmemorybench.com。
原文摘要 · Abstract (English)
Robots deployed in realistic settings will accumulate experience across many sessions and tasks over their deployment. The robot's tasks may often require it to remember information from multiple sessions ago, making long-context robot memory important for real-world deployments. However, most robot-memory benchmarks today are based on single episodes or a short context. To measure how current robot memory systems perform on longer sessions with more distractions, we introduce RoboMME-Interference, a cross-session benchmark built on RoboMME (Dai et al., 2026). For each query episode, we construct a session history using the query's relevant prior demonstration followed by a controlled number of unrelated sessions, which we provide to the VLA as memory and measure accuracy. Running RoboMME's released memory-augmented $π_{0.5}$ variants unmodified through this benchmark, we find that while perceptual memory variants improve success when given the history without any distractors, they decay strongly and steadily as unrelated sessions accumulate. The subgoal variants, which read the history with a vision-language model and pass written subgoals to the policy, improve less at their best but hold more of that improvement as distractors accumulate. Adding a retrieval step to the strongest perceptual variant, which selects the section of history most visually similar to the robot's current view and passes only that section to the policy, restores its no-distractor success rate at every interference level. With this release, we emphasize the importance of long-context memory and robustness to interference and show that current systems largely fail on such capabilities. The project page, videos, code, and data are at https://robotmemorybench.com.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。