arXiv:2411.18133cs.ROcs.CV2024-11被引 1

跨设备3D抓取新策略,无需重新训练即可适配不同品牌机械臂与相机。

3D-CDRGP: Towards Cross-Device Robotic Grasping Policy in 3D Open World

  • 用聚类生成3D物体候选框,结合二值语义提示提升开放世界检测能力。
  • 在两个不同厂商设备上验证,抓取成功率显著优于传统方法。
  • 适合做跨设备机器人抓取研究的开发者,尤其关注开放世界部署。

面对设备多样性和产品迭代加速,跨设备研究成为迫切需求。本文首次探索3D开放世界中跨设备(摄像头与机器人)的抓取策略。我们构建了两套真实场景下的抓取系统,使用来自不同制造商的机械臂和摄像头。为减少异构摄像头点云间的域差异,采用聚类方法生成3D物体候选框。然而,现有聚类方法局限于封闭集场景,限制了可抓取物体类别并固化部署环境。为此,我们提出SSGC-Seg模块,实现无类别依赖的3D物体检测:通过分析每个点的SoftMax值,将多分类语义转化为前景/背景二值提示,并基于几何信息对前景点聚类形成初始候选框。同时设计ScoreNet‡对每项检测结果评分,机器人优先抓取置信度最高的目标。在两种不同设备配置下的实验验证了该策略的有效性与鲁棒性。代码将于论文接受后公开。

原文摘要 · Abstract (English)

Given the diversity of devices and the product upgrades, cross-device research has become an urgent issue that needs to be tackled. To this end, we pioneer in probing the cross-device (cameras & robotics) grasping policy in the 3D open world. Specifically, we construct two real-world grasping setups, employing robotic arms and cameras from completely different manufacturers. To minimize domain differences in point clouds from diverse cameras, we adopt clustering methods to generate 3D object proposals. However, existing clustering methods are limited to closed-set scenarios, which confines the robotic graspable object categories and ossifies the deployment scenarios. To extend these methods to open-world settings, we introduce the SSGC-Seg module that enables category-agnostic 3D object detection. The proposed module transforms the original multi-class semantic information into binary semantic cues-foreground and background by analyzing the SoftMax value of each point, and then clusters the foreground points based on geometric information to form initial object proposals. Furthermore, ScoreNet‡ is designed to score each detection result, and the robotic arm prioritizes grasping the object with the highest confidence score. Experiments on two different types of setups highlight the effectiveness and robustness of our policy for cross-device robotics grasping research. Our code is provided in the supplementary and will be released upon acceptance.

3D抓取跨设备开放世界机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。