1000小时真实场景人手操作数据，提升机器人抓取成功率至60%
World In Your Hands: A Large-Scale and Open-Source Ecosystem for Learning Human-Centric Manipulation in the Wild
- 采集百万级动作数据，毫米级精度捕捉真实操作
- 在杂乱环境中机器人抓取成功率从8%提升至60%
- 适合研究人机协作与跨平台机械臂学习的团队
我们提出World In Your Hands（WIYH），一个大规模开源生态系统，包含超过1000小时在真实环境中采集的人类操作数据，具备毫米级运动精度。该系统包括：(1) Oracle Suite，一套可穿戴数据采集设备与自动标注流程；(2) WIYH数据集，涵盖数百种技能、上千小时多模态操作数据；(3) 丰富的标注与基准，支持从感知到动作的任务。基于该生态的实验表明，融合人类中心数据后，机器人在杂乱场景中的操作成功率从8%提升至60%。所有数据与硬件设计将开源。
原文摘要 · Abstract (English)
We introduce World In Your Hands (WIYH), a large-scale open-source ecosystem comprising over 1,000 hours of human manipulation data collected in-the-wild with millimeter-scale motion accuracy. Specifically, WIYH includes (1) the Oracle Suite, a wearable data collection kit with an auto-labeling pipeline for accurate motion capture; (2) the WIYH Dataset, featuring over 1,000 hours of multimodal manipulation data across hundreds of skills in diverse real-world scenarios; and (3) extensive annotations and benchmarks supporting tasks from perception to action. Furthermore, experiments based on the WIYH ecosystem show that integrating WIYH's human-centric data improves robotic manipulation success rates from 8% to 60% in cluttered scenes. World In Your Hands provides a foundation for advancing human-centric data collection and cross-embodiment policy learning. All data and hardware design will be open-source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。