用低成本视频演示实现多机器人灵巧操作,训练效率提升55.3%。
EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations

- 仅用单个RGB-D相机捕捉手势,通过MANO模型生成结构化演示数据
- 引入接触奖励动态调整策略,早期依赖演示,后期自主优化
- 在9种跨机器人配置下性能超基线55.3%,适合低成本灵巧操作研究
灵巧操作学习长期受限于数据与训练成本,纯强化学习需大规模交互探索,而模仿学习依赖高成本高质量演示。为此,我们提出EaDex框架,在低代价演示条件下实现多机器人灵巧操作学习,可快速生成演示数据并显著缩短训练时间。数据层面,EaDex仅使用单个RGB-D相机捕捉人手动作,结合MANO手部建模、数据归一化与运动重定向构建结构化演示数据。学习层面,提出基于接触奖励的动态演示退火机制,在初期引导探索,随接触奖励累积逐步转向自主优化。基于自建数据集,我们在三种灵巧手和三种可动物体开启任务上评估,覆盖九种跨机器人操作场景,相较无演示退火基线提升55.3%相对性能。结果验证了所提低成本演示流水线与动态演示退火策略的有效性。
原文摘要 · Abstract (English)
Dexterous manipulation learning has long been hindered by the high costs of data and training, as pure reinforcement learning typically requires large-scale interactive exploration and imitation learning depends on high-quality demonstrations that are expensive to collect. To address this problem, we propose EaDex, a multi-embodiment dexterous manipulation learning framework under low-cost demonstration conditions, which enables rapid generation of demonstration data and consequently reduces training time for efficient dexterous manipulation. At the data level, EaDex captures human hand motions using only a single RGB-D camera and constructs structured demonstration data through MANO-based hand modeling, data normalization, and motion retargeting. At the learning level, we introduce a contact-reward-based dynamic demonstration annealing mechanism, which guides early-stage exploration under demonstration and gradually transitions to autonomous optimization with accumulating contact rewards. Using our custom dataset, we evaluate EaDex on three dexterous hands and three articulated object-opening tasks, covering nine cross-embodiment manipulation settings, achieving a 55.3% relative improvement over the baseline without demonstration annealing. These results validate the effectiveness of the proposed low-cost demonstration pipeline and the dynamic demonstration annealing strategy for dexterous manipulation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。