arXiv:2510.14830cs.ROcs.AI2025-10被引 60

用强化学习让机器人在真实世界中稳定完成100次操作,成功率100%。

RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning

  • 融合模仿与强化学习,用去噪过程优化策略,提升稳定性。
  • 单个策略零样本成功率90%,连续任务成功率达100%。
  • 轻量化设计支持高频控制,适合工厂、家庭等真实场景。

现实世界中的机器人操作需具备接近甚至超越熟练人类的可靠性、效率和鲁棒性。本文提出RL-100,一种基于扩散视觉运动策略的真实世界强化学习框架。该框架在去噪过程中统一使用裁剪的PPO代理目标,实现离线与在线阶段的保守且稳定的性能提升。为满足部署延迟要求,采用轻量级一致性蒸馏方法将多步扩散压缩为单步控制器,支持高频控制。该框架对任务、机体和表征均无特定依赖,支持单动作与动作分块控制。我们在8个不同真实机器人任务上评估:动态推移、敏捷保龄球、倒液、折叠布料、拧开、多阶段榨汁及长时程盒子折叠。所有任务总1000次试验均成功(1000/1000),其中一项任务连续250次成功。性能达到或超过专家遥控操作员的时间效率。无需重训练,单一策略在环境与动力学变化下实现约90%零样本成功率;在少量样本下适应显著任务变化(86.7%);并能承受剧烈人为干扰(约96%)。值得注意的是,其榨汁机器人在商场零样本部署后连续运行约7小时无故障。这些结果表明,从人类先验出发,对齐训练目标与人类基准,可实现可部署的机器人学习。

原文摘要 · Abstract (English)

Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass those of skilled human operators. We present RL-100, a real-world reinforcement learning framework built on diffusion visuomotor policies. RL-100 unifies imitation and reinforcement learning under a single clipped PPO surrogate objective applied within the denoising process, yielding conservative and stable improvements across offline and online stages. To meet deployment latency requirements, a lightweight consistency distillation method compresses multi-step diffusion into a one-step controller for high-frequency control. The framework is task-, embodiment-, and representation-agnostic, and supports both single-action and action-chunking control. We evaluate RL-100 on eight diverse real-robot tasks, from dynamic pushing and agile bowling to pouring, cloth folding, unscrewing, multi-stage juicing, and long-horizon box folding. RL-100 attains 100 percent success across evaluated trials, for a total of 1000 out of 1000 episodes, including up to 250 out of 250 consecutive trials on one task. It matches or surpasses expert teleoperators in time to completion. Without retraining, a single policy attains approximately 90 percent zero-shot success under environmental and dynamics shifts, adapts in a few-shot regime to significant task variations (86.7 percent), and remains robust to aggressive human perturbations (about 96 percent). Notably, our juicing robot served random customers continuously for about seven hours without failure when deployed zero-shot in a shopping mall. These results suggest a practical path to deployment-ready robot learning by starting from human priors, aligning training objectives with human-grounded metrics, and reliably extending performance beyond human demonstrations.

强化学习机器人操控真实世界扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。