arXiv:2411.06183cs.RO2024-11被引 6

用采样MPC实现仿生腱驱动手的灵巧操作,无需训练即可快速适应任务。

Sampling-Based Model Predictive Control for Dexterous Manipulation on a Biomimetic Tendon-Driven Hand

  • 通过物理仿真器建模,采样MPC生成复杂接触行为
  • 每轮优化不到两分钟,实现在手滚动、翻转和抓取
  • 结合视觉语言模型动态调整目标函数,适合真实硬件部署

仿生柔性机器人手具备类人灵巧性潜力,但高维状态、复杂接触交互及状态估计不确定性使其控制困难。基于采样的模型预测控制(MPC)利用物理仿真器作为动力学模型,是生成丰富接触行为的有前景方法。然而,该方法尚未在真实(非仿真)机器人手上验证,尤其针对存在状态不确定性的柔性手。本文首次成功演示了在物理仿生腱驱动机器人手上使用采样MPC完成在手操作。尽管采样MPC无需强化学习的长期训练,但仍需针对具体任务调整目标函数以确保硬件上鲁棒执行。为此,我们将视觉语言模型(VLM)与实时优化器(MuJoCo MPC)结合:向VLM提供任务的高层人类语言描述和手部当前行为视频,由其逐步优化目标函数,每次迭代耗时不足两分钟。我们展示了在模拟和真实机器人手上实现球体滚动、翻转和抓取的可行性。结果表明,采样MPC是一种无需大量训练周期的高效方法,适用于仿生手的灵巧操作生成。

原文摘要 · Abstract (English)

Biomimetic and compliant robotic hands offer the potential for human-like dexterity, but controlling them is challenging due to high dimensionality, complex contact interactions, and uncertainties in state estimation. Sampling-based model predictive control (MPC), using a physics simulator as the dynamics model, is a promising approach for generating contact-rich behavior. However, sampling-based MPC has yet to be evaluated on physical (non-simulated) robotic hands, particularly on compliant hands with state uncertainties. We present the first successful demonstration of in-hand manipulation on a physical biomimetic tendon-driven robot hand using sampling-based MPC. While sampling-based MPC does not require lengthy training cycles like reinforcement learning approaches, it still necessitates adapting the task-specific objective function to ensure robust behavior execution on physical hardware. To adapt the objective function, we integrate a visual language model (VLM) with a real-time optimizer (MuJoCo MPC). We provide the VLM with a high-level human language description of the task and a video of the hand's current behavior. The VLM gradually adapts the objective function, allowing for efficient behavior generation, with each iteration taking less than two minutes. We show the feasibility of ball rolling, flipping, and catching using both simulated and physical robot hands. Our results demonstrate that sampling-based MPC is a promising approach for generating dexterous manipulation skills on biomimetic hands without extensive training cycles.

灵巧操作模型预测控制仿生机器人视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。