arXiv:2502.09180cs.RO2025-02

用机器学习让无触觉的机器人学会推物,效果媲美带触觉的同类

A Machine Learning Approach to Sensor Substitution from Tactile Sensing to Visual Perception for Non-Prehensile Manipulation

  • 训练模型将激光或视觉数据映射为缺失的触觉信息
  • 仅用激光雷达或RGB-D相机的机器人推物表现接近甚至超越触觉机器人
  • 适合传感器受限或需跨平台协作的机器人系统

移动操作机器人在复杂环境中应用日益广泛,需多种传感器感知与交互。但为每台机器人配备所有传感器不现实,尤其当不同机器人传感器配置差异大时。例如,一台配备高分辨率触觉皮肤的机器人擅长非抓取操作(如推动物体),若替换为无触觉传感器的机器人,则原有操控策略失效。本文提出一种基于机器学习的传感器替代框架,使仅有有限传感器(如激光雷达或RGB-D)的机器人能有效执行原本依赖丰富传感器(如触觉皮肤)的任务。该方法学习现有传感器数据与缺失传感器信息之间的映射,实现对缺失感官输入的合成。实验表明,通过训练,仅使用激光雷达或RGB-D的机器人在非预握式推物任务中表现可媲美甚至优于直接使用触觉反馈的机器人。

原文摘要 · Abstract (English)

Mobile manipulators are increasingly deployed in complex environments, requiring diverse sensors to perceive and interact with their surroundings. However, equipping every robot with every possible sensor is often impractical due to cost and physical constraints. A critical challenge arises when robots with differing sensor capabilities need to collaborate or perform similar tasks. For example, consider a scenario where a mobile manipulator equipped with high-resolution tactile skin is skilled at non-prehensile manipulation tasks like pushing. If this robot needs to be replaced or augmented by a robot lacking such tactile sensing, the learned manipulation policies become inapplicable. This paper addresses the problem of sensor substitution in non-prehensile manipulation. We propose a novel machine learning-based framework that enables a robot with a limited sensor set (e.g., LiDAR or RGB-D) to effectively perform tasks previously reliant on a richer sensor suite (e.g., tactile skin). Our approach learns a mapping between the available sensor data and the information provided by the substituted sensor, effectively synthesizing the missing sensory input. Specifically, we demonstrate the efficacy of our framework by training a model to substitute tactile skin data for the task of non-prehensile pushing using a mobile manipulator. We show that a manipulator equipped only with LiDAR or RGB-D can, after training, achieve comparable and sometimes even better pushing performance to a mobile base utilizing direct tactile feedback.

传感器替代触觉合成非抓取操作多模态学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。