arXiv:2607.11481cs.RO2026-07

让机器人手像人一样灵活操控物体,实现高成功率的远程精细操作。

Towards Human-level Dexterous Teleoperation

论文配图:Towards Human-level Dexterous Teleoperation
图 1 · 摘自论文原文
  • 通过共追踪控制器将操作者意图转化为低层接触动作,学习人类自然交互模式。
  • 在7个复杂任务中平均成功率75%,所有基线方法均失败。
  • 无需微调即可零样本迁移至真实机器人,适合工业级灵巧操作应用。

人类能流畅地使用工具、切换抓握方式并移动物体,无缝完成包含平移、重定向和指节运动的接触转换。当前的遥操作系统尚未具备这种级别的灵巧性。为此,本文提出TeleDexter,一种手-物协同追踪控制器,将操作者意图映射为学习到的低层接触执行策略。控制器基于人类参考动作生成连续的共追踪子目标进行训练,采用混合奖励机制(稀疏子目标与密集跟踪奖励结合),可在多种交互模式下学习,而非逐帧模仿轨迹。整个流程仅需单阶段强化学习,并通过随机动作掩码和领域随机化实现零样本迁移到真实机器人。我们在两种灵巧手上的7个挑战性任务中评估,涵盖物体重定向与长时程工具使用,平均成功率达75%,所有基线均失败。此外,收集的示范数据可直接用于行为克隆训练自主策略,标志着向人类级灵巧遥操作迈出关键一步。

原文摘要 · Abstract (English)

Humans routinely wield tools, swap grasps, and reposition objects within a single hand, seamlessly orchestrating contact transitions that span translation, reorientation, and finger gaiting. Endowing robot dexterous hands with this level of in-hand dexterity through teleoperation requires precise control of object motion via dynamic hand-object contact, yet current teleoperation systems remain far from this capability. To bridge this gap, we take a major step towards human-level dexterous teleoperation by introducing TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution. The controller is trained on consecutive co-tracking subgoals derived from human reference motions, utilizing a hybrid reward that couples sparse subgoal objectives with dense tracking rewards to enable learning across diverse interaction modalities rather than frame-wise trajectory imitation. The entire pipeline requires only single-stage RL and, with random action masking and domain randomization, transfers zero-shot to the real robot. We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands, achieving a 75% average success rate where all baselines consistently fail. Furthermore, the collected demonstrations successfully train autonomous policies via behavioral cloning, marking a concrete step towards human-level dexterous teleoperation.

灵巧操作遥操作强化学习机器人手

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。