通过前后场景对比，用视觉方法识别机器人搬运任务。
Scene Understanding in Pick-and-Place Tasks: Analyzing Transformations Between Initial and Final Scenes
- 基于物体边界框变化，提出几何与卷积神经网络两种检测方法。
- 基于VGG16的CNN方法在特定场景下准确率比几何法高12个百分点。
- 适合研究机器人环境理解、人机协作任务识别的研究者参考。
随着机器人在日常任务中越来越多地与人类协作，构建能够理解环境的机器人系统至关重要。本文聚焦于场景理解，旨在通过初始和最终场景图像识别拾取与放置任务。为此，收集了一个用于物体检测和拾取放置任务检测的数据集。使用YOLOv5网络对初始和最终场景中的物体进行检测。基于检测到的物体及其边界框，提出了两种方法来识别将初始场景转变为最终场景的拾取与放置任务。一种几何方法通过追踪两个场景中物体的移动，并依据边界框的交集判断运动；另一种基于CNN的方法利用卷积神经网络对有交集的边界框进行分类，划分为5类空间关系，从而推断出实际执行的任务。实验结果表明，在某些场景下，基于VGG16主干网络的CNN方法比几何方法高出约12个百分点，总体成功率达到84.3%。
原文摘要 · Abstract (English)
With robots increasingly collaborating with humans in everyday tasks, it is important to take steps toward robotic systems capable of understanding the environment. This work focuses on scene understanding to detect pick and place tasks given initial and final images from the scene. To this end, a dataset is collected for object detection and pick and place task detection. A YOLOv5 network is subsequently trained to detect the objects in the initial and final scenes. Given the detected objects and their bounding boxes, two methods are proposed to detect the pick and place tasks which transform the initial scene into the final scene. A geometric method is proposed which tracks objects' movements in the two scenes and works based on the intersection of the bounding boxes which moved within scenes. Contrarily, the CNN-based method utilizes a Convolutional Neural Network to classify objects with intersected bounding boxes into 5 classes, showing the spatial relationship between the involved objects. The performed pick and place tasks are then derived from analyzing the experiments with both scenes. Results show that the CNN-based method, using a VGG16 backbone, outscores the geometric method by roughly 12 percentage points in certain scenarios, with an overall success rate of 84.3%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。