用视觉识别抓取是否成功，尤其适合柔软物体。
Sim2Real Transfer for Vision-Based Grasp Verification
- 先用YOLO定位机械臂，再用ResNet判断物体是否被抓牢。
- 在真实环境中抓取验证准确率高，优于零样本问答基线。
- 自建合成数据集提升泛化能力，适合机器人抓取系统集成。
抓取成功验证是机器人操作中的关键环节，尤其在处理柔性物体时更具挑战。传统依赖力觉与触觉传感器的方法对非刚性物体效果有限。本文提出一种基于视觉的抓取验证方法：首先使用基于YOLO的目标检测模型定位机械臂，再通过基于ResNet的分类器判断物体是否存在。为克服真实数据采集的局限，我们构建了名为HSR-GraspSynth的合成数据集，用于模拟多样化的抓取场景。同时，将视觉问答能力作为零样本基线进行对比。实验结果表明，该方法在真实环境中表现出高准确性，具备集成至抓取流程的潜力。代码与数据集已公开于https://github.com/pauamargant/HSR-GraspSynth。
原文摘要 · Abstract (English)
The verification of successful grasps is a crucial aspect of robot manipulation, particularly when handling deformable objects. Traditional methods relying on force and tactile sensors often struggle with deformable and non-rigid objects. In this work, we present a vision-based approach for grasp verification to determine whether the robotic gripper has successfully grasped an object. Our method employs a two-stage architecture; first YOLO-based object detection model to detect and locate the robot's gripper and then a ResNet-based classifier determines the presence of an object. To address the limitations of real-world data capture, we introduce HSR-GraspSynth, a synthetic dataset designed to simulate diverse grasping scenarios. Furthermore, we explore the use of Visual Question Answering capabilities as a zero-shot baseline to which we compare our model. Experimental results demonstrate that our approach achieves high accuracy in real-world environments, with potential for integration into grasping pipelines. Code and datasets are publicly available at https://github.com/pauamargant/HSR-GraspSynth .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。