用合成数据训练双臂机器人通用灵巧抓取，零样本迁移到真实世界成功率超81%。
UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data
- 通过优化与规划结合生成高质量抓取轨迹，构建2000万帧多策略数据集
- 仅在仿真数据上训练的策略实现81.2%真实世界抓取成功率，可泛化到新物体
- 开源全流程数据生成工具，推动双臂灵巧操作研究
抓取是机器人与物理世界交互的基础能力。人类凭借双手能根据物体形状、大小和重量自主选择合适的抓取策略，实现稳定抓取与后续操作。相比之下，当前机器人抓取仍受限，尤其在多策略场景中。尽管平行夹爪和单手抓取已有较多研究，双臂灵巧抓取仍缺乏系统探索，数据是主要瓶颈。实现物理合理且几何契合、能承受外部力矩的抓取极具挑战。为此，我们提出UltraDexGrasp框架，支持双臂机器人的通用灵巧抓取。该框架融合基于优化的抓取生成与基于规划的示范生成，生成多样化的高质量抓取轨迹。基于此，我们构建了包含1000个物体、2000万帧数据的UltraDexGrasp-20M数据集。在此基础上，开发了一种简单有效的抓取策略：以点云为输入，通过单向注意力聚合场景特征，预测控制指令。该策略仅在合成数据上训练，即可实现稳健的零样本仿真到现实迁移，在真实世界对不同形状、大小和重量的新物体均表现良好，平均抓取成功率达81.2%。为促进未来双臂抓取研究，我们已开源数据生成管道（https://github.com/InternRobotics/UltraDexGrasp）。
原文摘要 · Abstract (English)
Grasping is a fundamental capability for robots to interact with the physical world. Humans, equipped with two hands, autonomously select appropriate grasp strategies based on the shape, size, and weight of objects, enabling robust grasping and subsequent manipulation. In contrast, current robotic grasping remains limited, particularly in multi-strategy settings. Although substantial efforts have targeted parallel-gripper and single-hand grasping, dexterous grasping for bimanual robots remains underexplored, with data being a primary bottleneck. Achieving physically plausible and geometrically conforming grasps that can withstand external wrenches poses significant challenges. To address these issues, we introduce UltraDexGrasp, a framework for universal dexterous grasping with bimanual robots. The proposed data-generation pipeline integrates optimization-based grasp synthesis with planning-based demonstration generation, yielding high-quality and diverse trajectories across multiple grasp strategies. With this framework, we curate UltraDexGrasp-20M, a large-scale, multi-strategy grasp dataset comprising 20 million frames across 1,000 objects. Based on UltraDexGrasp-20M, we further develop a simple yet effective grasp policy that takes point clouds as input, aggregates scene features via unidirectional attention, and predicts control commands. Trained exclusively on synthetic data, the policy achieves robust zero-shot sim-to-real transfer and consistently succeeds on novel objects with varied shapes, sizes, and weights, attaining an average success rate of 81.2% in real-world universal dexterous grasping. To facilitate future research on grasping with bimanual robots, we open-source the data generation pipeline at https://github.com/InternRobotics/UltraDexGrasp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。