开源15种机器人采集的18万条双手操作数据集,推动机器人协作研究。
RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation
- 构建跨平台双手操作数据集,覆盖16种环境与421项任务。
- 包含18万条示范数据,支持从轨迹到帧级的多粒度标注。
- 提供自动化处理工具链,适合多机器人协作研究者使用。
尽管双手操作对赋予机器人类人灵巧性至关重要,但由于双臂机器人平台间硬件差异大,大规模、多样化的数据集仍然稀缺。为弥补这一空白,我们推出RoboCOIN,一个大规模多实体双手操作数据集,包含来自15种不同机器人平台的超过18万条示范数据。数据覆盖16种多样化环境——包括住宅、商业和工业场景,涵盖421项双手操作任务,按39种协作动作和432种物体系统分类。本工作关键创新在于层级能力金字塔,实现从轨迹级概念到片段级子任务及帧级运动学的细粒度标注。此外,我们提出CoRobot数据处理流水线,基于机器人轨迹标记语言(RTML),可实现质量评估、自动标注与统一多实体数据管理。大量实验表明,RoboCOIN能有效提升多种双手操作模型在广泛机器人形态下的性能。整个数据集与代码库完全开源,为双手与多实体操作研究提供宝贵资源。
原文摘要 · Abstract (English)
Despite the critical role of bimanual manipulation in endowing robots with human-like dexterity, large-scale and diverse datasets remain scarce due to the significant hardware heterogeneity across bimanual robotic platforms. To bridge this gap, we introduce RoboCOIN, a large-scale multi-embodiment bimanual manipulation dataset comprising over 180,000 demonstrations collected from 15 distinct robotic platforms. Spanning 16 diverse environments-including residential, commercial, and industrial settings-the dataset features 421 bimanual tasks systematically categorized by 39 bimanual collaboration actions and 432 objects. A key innovation of our work is the hierarchical capability pyramid, which provides granular annotations ranging from trajectory-level concepts to segment-level subtasks and frame-level kinematics. Furthermore, we present CoRobot, an efficient data processing pipeline powered by the Robot Trajectory Markup Language (RTML), designed to facilitate quality assessment, automated annotation, and unified multi-embodiment and data management. Extensive experiments demonstrate the effectiveness of RoboCOIN in enhancing the performance of various bimanual manipulation models across a wide spectrum of robotic embodiments. The entire dataset and codebase are fully open-sourced, providing a valuable resource for advancing research in bimanual and multi-embodiment manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。