arXiv:2510.25754cs.RO2025-10

让机器人学会从多个物体中选最优工具,提升真实场景适应力。

GET-USE: Learning Generalized Tool Usage for Bimanual Mobile Manipulation via Simulated Embodiment Extensions

  • 通过仿真扩展机械臂末端,学习通用工具形状知识
  • 在3个任务中成功率提升30%-60%,优于现有方法
  • 适合需要灵活使用日常物品的机器人应用

当前机器人在通用化使用随机物体作为工具方面仍显不足,限制了其灵活性与问题解决能力。现有方法通常依赖为特定任务生成或众包获取工具数据集,但仅假设提供单一可操作物体,且无法在多个候选物中识别并使用最优工具,尤其当理想工具缺失时表现不佳。本文提出GeT-USE,一种两阶段方法:先在仿真中学习扩展机器人本体(如构建新末端执行器),识别对任务最有益的通用工具几何形态;再将此几何知识提炼为策略,指导真实机器人选择并使用最佳可用物体作为工具。在具有22个自由度的真实机器人上,该方法在三个基于视觉的双臂移动操作工具使用任务中,成功率相较现有方法提升30%至60%。

原文摘要 · Abstract (English)

The ability to use random objects as tools in a generalizable manner is a missing piece in robots' intelligence today to boost their versatility and problem-solving capabilities. State-of-the-art robotic tool usage methods focused on procedurally generating or crowd-sourcing datasets of tools for a task to learn how to grasp and manipulate them for that task. However, these methods assume that only one object is provided and that it is possible, with the correct grasp, to perform the task; they are not capable of identifying, grasping, and using the best object for a task when many are available, especially when the optimal tool is absent. In this work, we propose GeT-USE, a two-step procedure that learns to perform real-robot generalized tool usage by learning first to extend the robot's embodiment in simulation and then transferring the learned strategies to real-robot visuomotor policies. Our key insight is that by exploring a robot's embodiment extensions (i.e., building new end-effectors) in simulation, the robot can identify the general tool geometries most beneficial for a task. This learned geometric knowledge can then be distilled to perform generalized tool usage tasks by selecting and using the best available real-world object as tool. On a real robot with 22 degrees of freedom (DOFs), GeT-USE outperforms state-of-the-art methods by 30-60% success rates across three vision-based bimanual mobile manipulation tool-usage tasks.

机器人操作工具使用仿真迁移多目标选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。