用深度相机和图像联合估算物体质量,提升机器人抓取效率。
Estimating Object Physical Properties from RGB-D Vision and Depth Robot Sensors Using Deep Learning
- 融合RGB图像与稀疏点云数据,通过深度学习预测物体质量。
- 在合成数据上训练模型,使估计误差比基线降低37.5%以上。
- 适合做机器人感知与操作的工程师和研究者参考。
惯性质量在机器人抓取、操作和仿真中至关重要,能为规划与控制提供有力先验。在交互前准确估计物体质量,可显著提升多种任务性能。然而,仅用视觉传感器进行质量估计仍属探索不足领域。本文提出一种新方法,结合深度图像生成的稀疏点云与RGB图像,实现物体质量估计。我们评估了多种点云处理架构及仅使用RGB的方法。为解决训练数据稀缺问题,利用ShapeNetSem 3D模型,通过模拟Kinect相机生成合成RGBD图像,并训练一个图像生成模型以估计稠密深度图,进而扩充已有带质量标签的数据集。该方法在所有评测指标上均显著优于现有基准。数据生成(https://github.com/RavineWindteer/ShapenetSem-to-RGBD)、深度估计算法(https://github.com/RavineWindteer/GLPDepth-Edited)及质量估计算法(https://github.com/RavineWindteer/Depth-mass-estimator)均已公开。
原文摘要 · Abstract (English)
Inertial mass plays a crucial role in robotic applications such as object grasping, manipulation, and simulation, providing a strong prior for planning and control. Accurately estimating an object's mass before interaction can significantly enhance the performance of various robotic tasks. However, mass estimation using only vision sensors is a relatively underexplored area. This paper proposes a novel approach combining sparse point-cloud data from depth images with RGB images to estimate the mass of objects. We evaluate a range of point-cloud processing architectures, alongside RGB-only methods. To overcome the limited availability of training data, we create a synthetic dataset using ShapeNetSem 3D models, simulating RGBD images via a Kinect camera. This synthetic data is used to train an image generation model for estimating dense depth maps, which we then use to augment an existing dataset of images paired with mass values. Our approach significantly outperforms existing benchmarks across all evaluated metrics. The data generation (https://github.com/RavineWindteer/ShapenetSem-to-RGBD) as well as the training of the depth estimator (https://github.com/RavineWindteer/GLPDepth-Edited) and the mass estimator (https://github.com/RavineWindteer/Depth-mass-estimator) are available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。