Teleopit实现全身体感遥控人形机器人，手部动作自动适配不同机械手。
Teleopit: A Full-Embodiment Humanoid Teleoperation System
- 通过VR捕捉全身、手部和头部信号，实时映射至人形机器人
- 96次成功示范训练的策略在真实机器人上达到90%~95%任务成功率
- 无需调参即可适配不同机械手，支持复杂操作与视角控制
人形机器人演示数据采集需要协调全身运动、连续灵巧手控及视角控制。现有系统或简化手部指令，或依赖专用穿戴传感器进行精细手部动作捕捉。本文提出Teleopit，一个全身体感遥操作系统，可将虚拟现实中的身体、手部与头部信号映射至人形机器人，配备可配置灵巧手和2自由度主动视觉模块。采用历史编码器与故障感知回溯采样，在动作捕捉和实时VR输入下均提升运动追踪性能。基于优化的手部重定向器结合归一化手指方向、指尖闭合度与拇指帧对齐，无需调整手部特定目标函数或求解器超参数，即可将人类手部动作映射到不同灵巧手。组件实验评估了追踪成功率与重定向行为，真实机器人遥操作验证了行走、操作与视角控制的协同能力。基于Teleopit收集的96次成功示范，训练的ACT与GR00T N1.7策略在部署于人形机器人时分别达到90.0%与95.0%的任务成功率。
原文摘要 · Abstract (English)
Humanoid teleoperation for demonstration collection requires coordinated whole-body motion, continuous dexterous hand control, and viewpoint control. Existing systems either simplify hand commands or depend on dedicated wearable sensors for fine-grained hand motion. We introduce Teleopit, a full-embodiment teleoperation system that maps body, hand, and head signals from VR to a humanoid body, configurable dexterous hands, and a 2-DoF active vision module. A history encoder and failure-aware rewind sampling improve the motion tracker on both motion-capture and live VR references. An optimization-based hand retargeter combines normalized finger directions, fingertip closure, and thumb-frame alignment to map human hand motion to different dexterous hands without tuning hand-specific objective or solver hyperparameters. Component experiments evaluate tracking success rate and retargeting behavior, while real-robot teleoperation demonstrates coordinated locomotion, manipulation, and viewpoint control. ACT and GR00T N1.7 policies trained on 96 successful demonstrations collected with Teleopit achieve task success rates of 90.0% and 95.0%, respectively, when deployed on the humanoid. The project page is available at https://botrunner64.github.io/teleopit-page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。