模块化方法先估姿态形状再抓取,比端到端更有效。
Object Pose and Shape Estimation for Grasping: Does it Work?

- 分步处理:先估计物体姿态与形状,再用对称采样生成抓取点。
- 模块方法在小物体上表现优于端到端,成功率更高。
- 结合视觉语言模型可实现语言控制抓取,适合智能机器人应用。
近年来,物体姿态与形状估计取得显著进展。基于编码器-解码器(如SAM3D、LRM、CRISP)和扩散模型(如InstantMesh、Zero123、SceneComplete)的方法展现出类别无关的形状编码能力和开集泛化性。本文探讨:当用于平行夹爪、7自由度抓取及单视角RGB(-D)图像输入时,这些估计方法是否已足够成熟,能通过反向对称采样超越端到端抓取合成方法?我们对比了先进的端到端方法与三种模块化方法——先对场景中所有物体进行姿态与形状估计,再生成抓取。实验表明,模块化方法在所有测试中均优于端到端方法,尤其在小物体上表现优异,而端到端方法则失效。模块方法效果依赖于姿态与形状估计精度,在杂乱场景中性能下降,暴露现有方法局限性。我们分析了两种不同估计方式(编码器-解码器与扩散模型)的失败模式与运行时间,并证明单视角姿态与形状估计可结合视觉语言模型,仅凭单视图RGB-D图像实现语言条件化抓取,性能接近当前最优基线LERF-TOGO。
原文摘要 · Abstract (English)
The problem of object pose and shape estimation has seen key advancements lately. Encoder-decoder (e.g., SAM3D, LRM, CRISP) and diffusion-based models (e.g., InstantMesh, Zero123, SceneComplete) have shown category-agnostic shape encoding capacity and open-set generalizability. In this work, we ask the question: Are the object pose and shape estimation methods mature enough, such that when used with antipodal grasp sampling, can outperform the end-to-end grasp synthesis methods? We explore this question in detail by scoping our study to parallel jaw grippers, 7-DoF grasps, and single-view RGB(-D) image as input. We implement and compare a state-of-the-art, end-to-end grasp synthesis method and three modular methods, which first estimate the object pose and shape for all objects in the scene, and generate grasps using antipodal sampling. We observe that the modular methods outperform the end-to-end method in all our experiments. The modular methods are able to synthesize plenty of grasps, even for small objects, where the end-to-end methods fail. The effectiveness of the modular methods is contingent on the accuracy of the pose and shape estimation, and suffers partial degradation in cluttered scenes - a limitation of the existing pose and shape estimation methods. We also analyze the failure modes and run-times for the three modular methods, which use two different ways of object pose and shape estimation: one based on an encoder-decoder model, while another a diffusion model. Finally, we demonstrate that the single-view object pose and shape estimation methods can be augmented with vision-language models to yield language-conditioned grasps from just single-view RGB-D image as input. We notice comparable performance to the state-of-the-art LERF-TOGO baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。