找出关键记忆锚点，让机器人持续学习时不遗忘旧技能
Memory Anchors for Continual Robot Learning
- 在新旧任务冲突区域识别出对记忆保持至关重要的少数经验
- 移除10%记忆锚点使遗忘率提升4.5倍，加入锚点可降忘63%
- 适合研究持续学习、机器人技能积累的科研与工程人员
在真实环境中部署的机器人政策应具备持续学习新任务而不遗忘旧行为的能力。常见方法是使用包含先前任务数据的回放缓冲区进行训练。尽管该缓冲区通常随机采样所有过往经验，我们发现其中一小部分经验在锚定旧性能方面起着关键作用，称为记忆锚点。这些锚点出现在新任务观察的表征与旧任务观察表征重叠，但任务要求相反动作的区域，例如熟悉物体需以新方式操作时。在这些区域重放旧数据对防止过去知识被破坏性覆盖至关重要。在LIBERO基准测试中，仅排除10%的记忆锚点，导致灾难性遗忘增加超过4.5倍；反之，向回放缓冲区添加记忆锚点可降低高冲突任务遗忘63%，并实现真实机器人上两个任务序列的成功持续学习。
原文摘要 · Abstract (English)
Robot policies deployed in the wild should have the capability to continually learn new tasks without forgetting existing behaviors. A common approach to combat such catastrophic forgetting is to train on new task data with a replay buffer of previously learned task data. Although this buffer is commonly sampled randomly from all prior experiences, we show that a small set of these experiences contributes greatly in anchoring past performance. We call these experiences Memory Anchors. We identify Memory Anchors in regions where representations of new-task observations collapse onto those of old-task observations even though the tasks require conflicting actions, like when a familiar object must be manipulated in a new way. Rehearsing old data in this region plays a key role in preventing destructive overwriting of past task knowledge, serving as this critical Memory Anchor role. Excluding only 10% Memory Anchors before sampling the buffer leads to more than a 4.5x increase in catastrophic forgetting on the LIBERO benchmark suites. Conversely, enriching the replay buffer with Memory Anchors can decrease high-conflict task forgetting by 63% and enables successful continual learning of two task sequences on a real robot. Videos and additional visualizations can be found at https://robot-adaptation.github.io/MemoryAnchors
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。