arXiv:2608.21899cs.RO2026-08

让机器人持续学新技能不丢旧本领,10分钟学会一项新操作。

CIDER: Continual Interactive Distillation for Embodied Reinforcement Learning

论文配图:CIDER: Continual Interactive Distillation for Embodied Reinforcement Learning
图 1 · 摘自论文原文
  • 用历史策略当教师,学习新任务时实时回放旧知识。
  • 六项真实任务连续训练,旧任务成功率保持90%以上。
  • 适合需要长期学习的机器人场景,如家庭或工业操作。

人机协同的真实世界强化学习可在数十分钟内快速获取单个任务的有效机器人操作策略。然而,如何将这一范式扩展到持续学习仍不明确——即单一策略需在不遗忘已有行为的前提下持续习得新技能。现有真实世界持续学习方法未显式约束先前行为,导致严重灾难性遗忘。本文提出面向具身强化学习的持续交互式蒸馏(CIDER),在学习每个新任务前冻结累积的历史策略作为教师,并在任务学习中穿插基于蒸馏的行为保留机制。进一步引入梯度路由,分离用于获取新任务与保留旧行为的梯度。我们在六个真实世界的家用与工业操作任务上,使用单一共享执行器评估该方法。交互式蒸馏在六项任务序列中均保持高成功率,且每项新任务仅需10至20分钟即可掌握,而所有基线方法均至少遗忘一个旧任务。额外消融实验揭示了决定真实世界持续强化学习中稳定性与可塑性权衡的关键设计选择。

原文摘要 · Abstract (English)

Human-in-the-loop real-world reinforcement learning enables rapid acquisition of effective robotic manipulation policies for individual tasks, often within tens of minutes. Yet it remains unclear how to extend this paradigm to continual learning, where a single policy must acquire new skills without losing previously learned behaviors. Existing real-world continual learning methods do not explicitly constrain prior behaviors, leading to severe catastrophic forgetting. We introduce Continual Interactive Distillation for Embodied Reinforcement Learning (CIDER), a continual reinforcement learning framework that freezes the accumulated historical policy as a teacher before learning each new task and interleaves task learning with distillation-based retention. We further introduce gradient routing to separate the gradients used for acquiring new tasks from those used for preserving prior behaviors. We evaluate our method with a single shared actor on six real-world household and industrial manipulation tasks. Interactive Distillation maintains high measured success on previously learned tasks across our six-task real-robot sequence while acquiring each new task in 10 to 20 minutes, whereas every baseline forgets at least one previous task. Additional ablations reveal the key design choices that govern the tradeoff between stability and plasticity in real-world continual reinforcement learning.

持续学习机器人控制强化学习蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。