从单视角点云学习多物体场景下的灵巧抓取,提升真实场景泛化能力。
End-to-End Dexterous Grasp Learning from Single-View Point Clouds via a Multi-Object Scene Dataset
- 端到端网络直接从单视角点云预测密集抓取姿态。
- 仿真与实机抓取成功率分别达88.63%和78.98%,穿透深度仅0.375mm。
- 适用于需要复杂环境适应的机器人抓取任务,尤其适合工业场景。
多物体场景中的灵巧抓取是机器人操作的核心挑战。现有主流抓取数据集多聚焦于单物体场景与预设抓取姿态,常忽略环境干扰及灵巧预抓姿势建模,限制了实际应用的泛化能力。为此,我们提出DGS-Net,一种可从多物体场景单视角点云中学习密集抓取配置的端到端抓取预测网络。同时,设计两阶段抓取数据生成策略:先生成密集单物体抓取,再扩展至场景级密集抓取。数据集包含307个物体、240个多物体场景及超过35万条有效抓取样本。通过显式建模抓取偏移量与预抓姿态,提供更鲁棒准确的监督信号。实验表明,DGS-Net在仿真中达到88.63%抓取成功率,在真实机器人平台达78.98%,平均穿透深度0.375 mm,穿透体积559.45 mm³,优于现有方法,展现出强有效性与泛化能力。数据集已开源:https://github.com/4taotao8/DGS-Net。
原文摘要 · Abstract (English)
Dexterous grasping in multi-object scene constitutes a fundamental challenge in robotic manipulation. Current mainstream grasping datasets predominantly focus on single-object scenarios and predefined grasp configurations, often neglecting environmental interference and the modeling of dexterous pre-grasp gesture, thereby limiting their generalizability in real-world applications. To address this, we propose DGS-Net, an end-to-end grasp prediction network capable of learning dense grasp configurations from single-view point clouds in multi-object scene. Furthermore, we propose a two-stage grasp data generation strategy that progresses from dense single-object grasp synthesis to dense scene-level grasp generation. Our dataset comprises 307 objects, 240 multi-object scenes, and over 350k validated grasps. By explicitly modeling grasp offsets and pre-grasp configurations, the dataset provides more robust and accurate supervision for dexterous grasp learning. Experimental results show that DGS-Net achieves grasp success rates of 88.63\% in simulation and 78.98\% on a real robotic platform, while exhibiting lower penetration with a mean penetration depth of 0.375 mm and penetration volume of 559.45 mm^3, outperforming existing methods and demonstrating strong effectiveness and generalization capability. Our dataset is available at https://github.com/4taotao8/DGS-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。