用双目视觉重建手物姿态，让机器人更准接住各种物品
Stereo Hand-Object Reconstruction for Human-to-Robot Handover
- 基于双目视觉融合单视角重建结果，提升三维结构一致性
- 在透明物体上表现优于现有方法，物体切比雪夫距离更低
- 适合需要精准抓取的机器人交互场景，如家庭服务
联合估计手部与物体形状有助于实现人机交接中的抓取任务。然而，依赖手工设计的物体几何先验在面对未见物体时泛化能力差，且深度传感器难以检测透明物体（如玻璃杯）。本文提出一种基于双目的手物重建方法，通过概率融合单视图重建结果生成一致的立体重建。我们利用大规模合成手物数据集学习3D形状先验，确保方法可泛化，并采用RGB输入以更好捕捉透明物体。实验表明，本方法在单视图和双视图设置下，相较于现有基于RGB的手物重建方法，显著降低了物体切比雪夫距离。通过基于投影的异常值剔除步骤处理重建结果，指导使用宽基线双目RGB相机的人机交接流程。该重建方法使机器人成功接收多种家用物品。
原文摘要 · Abstract (English)
Jointly estimating hand and object shape facilitates the grasping task in human-to-robot handovers. However, relying on hand-crafted prior knowledge about the geometric structure of the object fails when generalising to unseen objects, and depth sensors fail to detect transparent objects such as drinking glasses. In this work, we propose a stereo-based method for hand-object reconstruction that combines single-view reconstructions probabilistically to form a coherent stereo reconstruction. We learn 3D shape priors from a large synthetic hand-object dataset to ensure that our method is generalisable, and use RGB inputs to better capture transparent objects. We show that our method reduces the object Chamfer distance compared to existing RGB based hand-object reconstruction methods on single view and stereo settings. We process the reconstructed hand-object shape with a projection-based outlier removal step and use the output to guide a human-to-robot handover pipeline with wide-baseline stereo RGB cameras. Our hand-object reconstruction enables a robot to successfully receive a diverse range of household objects from the human.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。