用少量示范教机器人像服务员一样动态推物,还支持新物体和纠错。
Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning
- 用演示数据预训练价值函数,结合感知不确定性的模型预测控制
- 50-100次示范即可在仿真与真实机器人上稳定运行
- 适合想快速训练机器人做非抓取操作的研究者或工程师
我们研究如何让机械臂仅通过少量真实世界示范,完成动态非抓取物体搬运(即“机器人服务员”任务)。提出一种结合批量强化学习与模型预测控制的方法:先从示范数据预训练一组价值函数,再在线使用不确定性感知的模型预测控制框架,提升对有限数据覆盖的鲁棒性。该方法可无缝集成至现有MPC系统,仅需任务空间示范且过渡状态标签稀疏,同时借助MPC保证关节空间运动平滑与约束满足。我们在Franka Panda机器人上进行了大量仿真与真实实验,验证了该方法在50-100次示范下实现稳健部署。此外,模型能泛化到训练中未见的新物体,并可优化次优示范。我们认为该框架可显著降低示范需求,加速机器人非抓取操作任务的训练进程。项目视频与补充材料详见:https://sites.google.com/view/cvmpc。
原文摘要 · Abstract (English)
We investigate the problem of teaching a robot manipulator to perform dynamic non-prehensile object transport, also known as the `robot waiter' task, from a limited set of real-world demonstrations. We propose an approach that combines batch reinforcement learning (RL) with model-predictive control (MPC) by pretraining an ensemble of value functions from demonstration data, and utilizing them online within an uncertainty-aware MPC scheme to ensure robustness to limited data coverage. Our approach is straightforward to integrate with off-the-shelf MPC frameworks and enables learning solely from task space demonstrations with sparsely labeled transitions, while leveraging MPC to ensure smooth joint space motions and constraint satisfaction. We validate the proposed approach through extensive simulated and real-world experiments on a Franka Panda robot performing the robot waiter task and demonstrate robust deployment of value functions learned from 50-100 demonstrations. Furthermore, our approach enables generalization to novel objects not seen during training and can improve upon suboptimal demonstrations. We believe that such a framework can reduce the burden of providing extensive demonstrations and facilitate rapid training of robot manipulators to perform non-prehensile manipulation tasks. Project videos and supplementary material can be found at: https://sites.google.com/view/cvmpc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。