仅用彩色图像实现灵巧手抓取，无需深度图或物体姿态
DextrAH-RGB: Visuomotor Policies to Grasp Anything with Dexterous Hands
- 通过仿真训练几何布料控制器的优越策略，再蒸馏为纯RGB输入策略
- 在真实世界中成功抓取未见过的物体，涵盖新几何、纹理和光照条件
- 首个端到端RGB输入的灵巧抓取系统，适合机器人抓取研究者参考
灵巧机器人抓取多样物体是一项重要但极具挑战的任务。以往工作受限于速度、泛化性或依赖深度图与物体位姿。本文提出DextrAH-RGB，一个从彩色图像输入端到端实现灵巧机械臂-手抓取的系统。我们在仿真中通过强化学习训练一个基于几何布料控制器的优越策略（FGP），随后使用逼真拼贴渲染技术，在仿真中将该策略蒸馏为仅依赖RGB图像的FGP。据我们所知，这是首个在复杂、动态、接触密集任务（如灵巧抓取）中实现鲁棒的模拟到现实迁移的端到端RGB策略。DextrAH-RGB在性能上媲美基于深度图的灵巧抓取策略，并能泛化至真实世界中未见过的物体，包括未知几何、纹理和光照条件。系统抓取多样化未知物体的视频见：https://dextrah-rgb.github.io/
原文摘要 · Abstract (English)
One of the most important, yet challenging, skills for a dexterous robot is grasping a diverse range of objects. Much of the prior work has been limited by speed, generality, or reliance on depth maps and object poses. In this paper, we introduce DextrAH-RGB, a system that can perform dexterous arm-hand grasping end-to-end from RGB image input. We train a privileged fabric-guided policy (FGP) in simulation through reinforcement learning that acts on a geometric fabric controller to dexterously grasp a wide variety of objects. We then distill this privileged FGP into a RGB-based FGP strictly in simulation using photorealistic tiled rendering. To our knowledge, this is the first work that is able to demonstrate robust sim2real transfer of an end2end RGB-based policy for complex, dynamic, contact-rich tasks such as dexterous grasping. DextrAH-RGB is competitive with depth-based dexterous grasping policies, and generalizes to novel objects with unseen geometry, texture, and lighting conditions in the real world. Videos of our system grasping a diverse range of unseen objects are available at \url{https://dextrah-rgb.github.io/}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。