arXiv:2506.14317cs.RO2025-06中稿 · CoRL被引 21

无需真实数据,模拟训练实现复杂场景抓取的零样本部署。

ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes

  • 两阶段师生框架,模拟中通过密度课程学习提升泛化能力。
  • 在多种物体和布局下实现稳定抓取,成功率高于基线方法。
  • 适合机器人抓取、智能仓储等需要复杂操作的场景应用。

在杂乱场景中进行灵巧抓取面临物体几何多样、遮挡和碰撞等挑战。现有方法多聚焦单物体抓取或无交互的姿态预测,难以应对复杂环境。尽管视觉-语言-动作模型有潜力,但需大量真实演示,成本高且难扩展。为此,我们重新审视模拟到现实的迁移流程,提出关键技术实现零样本现实部署并保持强泛化性。设计了ClutterDexGrasp系统,采用两阶段教师-学生框架,支持闭环目标导向灵巧抓取。教师策略在模拟中通过杂乱密度课程学习,融合几何与空间嵌入场景表示,并引入新型综合安全课程,实现通用、动态、安全的抓取行为。通过模仿学习,将教师知识提炼为基于部分点云观测的3D扩散策略(DP3)。据我们所知,这是首个在杂乱场景中实现目标导向灵巧抓取的零样本模拟到现实闭环系统,在多样化物体与布局下表现稳健。更多细节及视频见https://clutterdexgrasp.github.io/。

原文摘要 · Abstract (English)

Dexterous grasping in cluttered scenes presents significant challenges due to diverse object geometries, occlusions, and potential collisions. Existing methods primarily focus on single-object grasping or grasp-pose prediction without interaction, which are insufficient for complex, cluttered scenes. Recent vision-language-action models offer a potential solution but require extensive real-world demonstrations, making them costly and difficult to scale. To address these limitations, we revisit the sim-to-real transfer pipeline and develop key techniques that enable zero-shot deployment in reality while maintaining robust generalization. We propose ClutterDexGrasp, a two-stage teacher-student framework for closed-loop target-oriented dexterous grasping in cluttered scenes. The framework features a teacher policy trained in simulation using clutter density curriculum learning, incorporating both a geometry and spatially-embedded scene representation and a novel comprehensive safety curriculum, enabling general, dynamic, and safe grasping behaviors. Through imitation learning, we distill the teacher's knowledge into a student 3D diffusion policy (DP3) that operates on partial point cloud observations. To the best of our knowledge, this represents the first zero-shot sim-to-real closed-loop system for target-oriented dexterous grasping in cluttered scenes, demonstrating robust performance across diverse objects and layouts. More details and videos are available at https://clutterdexgrasp.github.io/.

灵巧抓取模拟到现实扩散模型闭环控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。