让机器人在离线时巩固旧技能，避免遗忘且保持统一控制策略。
When Robots Sleep: Offline Skill Consolidation for Shared-Policy Robot Learning
- 用睡眠期离线整合新技能，通过冻结的记忆模块更新共享策略。
- 在Meta-World上成功率提升64%，相关技能可靠性翻倍。
- 适合长期部署的机器人持续学习，尤其需保持统一控制结构的场景。
长期部署的机器人需在不丢失共享策略结构的前提下新增技能。我们研究序列化机器人技能学习，其中历史轨迹和任务损失可能不可用，且部署策略必须为单一共享控制器，不能使用特定任务头、路由或适配器。我们发现技能耦合崩溃：个体技能仍有效，但相关技能间的可靠性下降。提出「沉睡机器人」框架，通过唤醒期学习新技能，睡眠期离线整合共享策略，使用紧凑的冻结技能记忆——用于强化学习的无序状态缓存冻结评价器，以及用于模仿学习的无序观察缓存冻结执行器快照。睡眠期间，这些记忆定义可微分的代理目标，通过纳什讨价还价组合梯度，并结合自适应锚定与局部兴奋性实现稳定整合。在Meta-World MT5上，相比最强非预言基线，平均成功率提升64%，成对可靠性提升2.0倍；在SurgicAI上，平均成功率与后向迁移性能优于持续模仿基线，同时保持良好的成对可靠性。
原文摘要 · Abstract (English)
Robots that learn over long deployments must add new skills without losing the shared policy structure that makes earlier skills reusable. We study sequential robot skill learning, where previous trajectories and task losses may be unavailable, and the deployed policy must remain a single shared controller without task-specific heads, routing, or adapters. We identify skill-coupling collapse, a failure mode in which individual skill success remains non-trivial while reliability among related skills deteriorates. We propose Sleeping Robots, a wake-sleep framework that learns each new skill during wake and consolidates the shared policy offline during sleep using compact frozen skill memories: frozen critics with unordered state buffers for reinforcement learning and frozen actor snapshots with unordered observation buffers for imitation learning. During sleep, these memories define differentiable surrogate objectives whose gradients are combined through Nash bargaining, with adaptive anchoring and local excitability for stable consolidation. On Meta-World MT5, Sleeping Robots improves average success by 64 % and pairwise reliability by x 2.0 over the strongest non-oracle baseline, and on SurgicAI it improves average success and backward transfer relative to continual imitation baselines while remaining competitive on pairwise reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。