低成本可复现的全身人形机器人数据采集系统，15分钟收集100次成功示范。
TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System
- 用头戴VR+自研2自由度机械颈实现全身动作捕捉与第一视角视觉
- 15分钟内完成100次示范，成功率接近100%，支持长时序精细操作
- 适合关注人形机器人数据集构建与自主控制的研究者
大规模数据推动了机器人领域的发展，从语言模型到多模态视觉-语言-动作模型。然而，人形机器人仍缺乏高效的数据采集框架。现有遥操作系统或采用解耦控制，或依赖昂贵的动作捕捉设备。我们提出TWIST2，一种便携、无需动捕的人形机器人遥操作与数据采集系统，在保持全身体控的同时提升可扩展性。系统利用PICO4U VR实时获取全身人类动作，并配备定制2自由度机器人颈部（成本约250美元）实现第一视角视觉，支持全身至人形机器人的协同控制。实验表明，系统可完成长时间精细且移动的操作技能，15分钟内完成100次示范，成功率接近100%。基于此流程，我们设计了一种分层视觉运动策略框架，基于第一视角视觉自主控制整个机器人身体。该策略成功实现了全身精细操作与动态踢击任务。整个系统完全可复现并开源于https://yanjieze.com/TWIST2，所采集数据集也已公开：https://twist-data.github.io。
原文摘要 · Abstract (English)
Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effective data collection frameworks. Existing humanoid teleoperation systems either use decoupled control or depend on expensive motion capture setups. We introduce TWIST2, a portable, mocap-free humanoid teleoperation and data collection system that preserves full whole-body control while advancing scalability. Our system leverages PICO4U VR for obtaining real-time whole-body human motions, with a custom 2-DoF robot neck (cost around $250) for egocentric vision, enabling holistic human-to-humanoid control. We demonstrate long-horizon dexterous and mobile humanoid skills and we can collect 100 demonstrations in 15 minutes with an almost 100% success rate. Building on this pipeline, we propose a hierarchical visuomotor policy framework that autonomously controls the full humanoid body based on egocentric vision. Our visuomotor policy successfully demonstrates whole-body dexterous manipulation and dynamic kicking tasks. The entire system is fully reproducible and open-sourced at https://yanjieze.com/TWIST2 . Our collected dataset is also open-sourced at https://twist-data.github.io .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。