arXiv:2507.01857cs.RO2025-07被引 12

让机器人手按动作类型操作,突破人类手势限制。

TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types

  • 用动作类型引导机械手操作,不依赖人类手势模仿。
  • 真实场景测试显示任务成功率显著提升。
  • 适合需要高灵巧性的远程机器人控制场景。

灵巧遥操作在现实世界数据采集和远程机器人控制中至关重要。以往方法主要依赖手部姿态重定向以模仿人类手势,但未能充分利用灵巧机械手的结构优势。为此,我们提出TypeTele系统,通过引入灵巧操作类型,使机械手能执行不受人类运动模式约束的独特动作。我们构建了一个可扩展的灵巧操作类型库,覆盖多种操作姿势。操作时,采用多模态大语言模型辅助的类型检索模块,根据任务需求和操作指令自动匹配最合适的动作类型。大量真实世界遥操作与模仿学习实验表明,引入操作类型显著提升了机械手完成多样化复杂任务的能力,成功率达更高。

原文摘要 · Abstract (English)

Dexterous teleoperation plays a crucial role in robotic manipulation for real-world data collection and remote robot control. Previous dexterous teleoperation mostly relies on hand retargeting to closely mimic human hand postures. However, these approaches may fail to fully leverage the inherent dexterity of dexterous hands, which can execute unique actions through their structural advantages compared to human hands. To address this limitation, we propose TypeTele, a type-guided dexterous teleoperation system, which enables dexterous hands to perform actions that are not constrained by human motion patterns. This is achieved by introducing dexterous manipulation types into the teleoperation system, allowing operators to employ appropriate types to complete specific tasks. To support this system, we build an extensible dexterous manipulation type library to cover comprehensive dexterous postures used in manipulation tasks. During teleoperation, we employ a MLLM (Multi-modality Large Language Model)-assisted type retrieval module to identify the most suitable manipulation type based on the specific task and operator commands. Extensive experiments of real-world teleoperation and imitation learning demonstrate that the incorporation of manipulation types significantly takes full advantage of the dexterous robot's ability to perform diverse and complex tasks with higher success rates.

遥操作灵巧手动作类型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。