用真人手当通用操控接口，让机械手学会精细操作。
DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation
- 用可穿戴外骨骼捕捉人手动作并适配机器人手
- 真实实验中任务成功率平均达86%
- 适合想快速训练机械手抓取的科研与工程人员
我们提出DexUMI——一个基于人类手部作为自然操控界面的数据采集与策略学习框架，用于将灵巧操作技能迁移至多种机器人手。DexUMI通过硬件与软件双重适配缩小人机差异：硬件方面采用可穿戴手部外骨骼，弥合运动学差距，支持直接触觉反馈，并将人手动作映射为机器人可行动作；软件方面通过高保真机器人手图像修复，替换视频中的人手以弥合视觉差距。我们在两种不同灵巧机器人手平台上开展全面实测，平均任务成功率达到86%。
原文摘要 · Abstract (English)
We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。