构建7000小时手物交互数据集，助力机器人精细操作学习
DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
- 基于真实人类演示生成合成数据，覆盖21类基础操作技能
- 每帧含高精度动作捕捉与物理一致的接触力信息
- 适用于机器人操作学习、接触控制及不同手型间技能迁移
我们提出DexCanvas，一个大规模混合真实-合成的人类操作数据集，包含7,000小时精细手物交互数据，源自70小时真实人类示范，按Cutkosky分类法划分为21种基础操作类型。每条记录融合同步多视角RGB-D图像、高精度动捕数据（含MANO手部参数）及每帧物理一致的接触点与受力特征。真实到仿真转化管道利用强化学习训练策略，在物理仿真中操控可驱动的MANO手，复现人类示范并推断产生物体运动的底层接触力。DexCanvas是首个结合大规模真实示范、基于成熟分类体系的系统性技能覆盖和物理验证接触标注的操纵数据集，可推动机器人操纵学习、接触密集控制及跨手型技能迁移研究。
原文摘要 · Abstract (English)
We present DexCanvas, a large-scale hybrid real-synthetic human manipulation dataset containing 7,000 hours of dexterous hand-object interactions seeded from 70 hours of real human demonstrations, organized across 21 fundamental manipulation types based on the Cutkosky taxonomy. Each entry combines synchronized multi-view RGB-D, high-precision mocap with MANO hand parameters, and per-frame contact points with physically consistent force profiles. Our real-to-sim pipeline uses reinforcement learning to train policies that control an actuated MANO hand in physics simulation, reproducing human demonstrations while discovering the underlying contact forces that generate the observed object motion. DexCanvas is the first manipulation dataset to combine large-scale real demonstrations, systematic skill coverage based on established taxonomies, and physics-validated contact annotations. The dataset can facilitate research in robotic manipulation learning, contact-rich control, and skill transfer across different hand morphologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。