arXiv:2603.17065cs.RO2026-03

用手机就能远程操控机械手,快速收集真实世界演示数据。

TeleDex: Accessible Dexterous Teleoperation

  • 用手机实时传输6-DoF手腕和21-DoF手部动作,无需外部追踪设备
  • 支持手持模式与3D打印手柄模式,实现手指级精细控制
  • 开源系统,适合机器人部署中快速收集新任务演示

尽管数据集规模和模型能力持续提升,当前机器人操作策略仍难以泛化至训练分布之外。因此,在新环境、新任务或新机器人形态中部署先进策略时,常需额外收集示范数据。实现实时部署中的这一需求,需要能快速、低成本、低门槛采集示范的工具。我们提出TeleDex,一个开源系统,仅使用任意现成手机即可直观操控灵巧手和机器人机械臂。系统通过手机流式传输低延迟的6-DoF手腕姿态和21-DoF关节状态估计,并直接映射至机器人手臂与多指机械手,无需外部追踪设施。TeleDex支持纯手持手机模式,也提供可3D打印的手持接口以实现指尖级操作。通过降低灵巧操作的软硬件门槛,TeleDex使用户可在部署过程中快速采集示范数据,用于策略微调。我们在仿真与真实世界操作任务中评估了该系统,验证其作为统一可扩展机器人遥操作接口的有效性。所有软件与硬件设计及演示视频均开源,地址为orayyan.com/teledex。

原文摘要 · Abstract (English)

Despite increasing dataset scale and model capacity, robot manipulation policies still struggle to generalize beyond their training distributions. As a result, deploying state-of-the-art policies in new environments, tasks, or robot embodiments often requires collecting additional demonstrations. Enabling this in real-world deployment settings requires tools that allow users to collect demonstrations quickly, affordably, and with minimal setup. We present TeleDex, an open-source system for intuitive teleoperation of dexterous hands and robotic manipulators using any readily available phone. The system streams low-latency 6-DoF wrist poses and articulated 21-DoF hand state estimates from the phone, which are retargeted to robot arms and multi-fingered hands without requiring external tracking infrastructure. TeleDex supports both a handheld phone-only mode and an optional 3D-printable hand-mounted interface for finger-level teleoperation. By lowering the hardware and setup barriers to dexterous teleoperation, TeleDex enables users to quickly collect demonstrations during deployment to support policy fine-tuning. We evaluate the system across simulation and real-world manipulation tasks, demonstrating its effectiveness as a unified scalable interface for robot teleoperation. All software and hardware designs, along with demonstration videos, are open-source and available at orayyan.com/teledex.

遥操作灵巧手手机控制开源系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。