让双臂机器人精准模仿人手动作并自适应抓取物体
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
- 通过视觉捕捉人体动作，跨平台精准转换为机器人运动
- 结合视觉语言模型与预测控制，实现物体抓握力自适应调节
- 支持多类型机器人和复杂物体抓取，适合人机协作场景
在双臂灵巧遥操作中，动作迁移的跨平台泛化能力与抓取交互性至关重要。然而，机器人结构差异及抓取物品种类繁多，给精确动作迁移与柔顺抓取带来挑战。为此，提出基于动作重定向与自适应力控的双臂灵巧遥操作系统（DexTele）。首先，设计基于视觉的动作重定向模块，利用动作图编码器与潜在空间优化，实现跨平台精准动作迁移。其次，设计自适应抓取模块，融合视觉语言模型（VLM）与模型预测控制（MPC），可预测目标物体所需抓握力，并进行梯度驱动在线优化。大量实验表明，DexTele实现了跨多机器人平台的精确动作迁移与柔顺抓取。
原文摘要 · Abstract (English)
In dual-arm dexterous teleoperation, cross-platform generalization of motion retargeting and interactivity of grasping are crucial. However, the heterogeneity of robotic architectures and the wide variety of grasping objects pose significant challenges to achieving precise motion retargeting and compliant grasping in dual-arm dexterous teleoperation. To address these challenges, a dual-arm dexterous teleoperation system (DexTele) is proposed based on motion retargeting and adaptive force control. First, a vision-based motion retargeting module is designed to generate preliminary robot motions from human images. In this module, a motion-graph encoder and latent optimization are proposed for precise and convenient cross-platform motion retargeting. Second, an adaptive grasping module is designed to achieve compliant grasping. This module combines a vision-language model (VLM) with model predictive control (MPC), allowing the system to predict the required grasping force for a target object and perform gradient-based online optimization. Finally, extensive experiments demonstrate that the DexTele achieves precise motion retargeting and compliant grasping with generalization across multiple robot platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。