arXiv:2507.13200cs.RO2025-07被引 6

用少量人类示范教机器人用工具,靠触觉与距离传感器提升泛化能力。

Few-shot transfer of tool-use skills using human demonstrations with proximity and tactile sensing

  • 用仿真预训练+真实微调,实现少样本技能迁移。
  • 仅需少量示范即可让机械臂学会不同工具的表面跟随任务。
  • 触觉与距离传感器融合提升接触状态识别精度。

工具扩展了机器人的操作能力,如同对人类一样。尽管人类擅长工具操作,但教会机器人完成这些技能仍面临挑战,主要源于机器人与工具、工具与环境之间两个同时存在的接触点。触觉与距离传感器在识别此类复杂接触中起关键作用。然而,由于真实世界数据有限且存在显著的仿真到现实差距,利用这些传感器学习工具操作仍具挑战性。为此,我们提出一种基于多模态感知的少样本工具使用技能迁移框架:先在仿真中预训练基础策略以捕捉工具操作中共有的接触状态,再通过真实目标域中收集的人类示范进行微调,以弥合域间差异。我们在Franka Emika机械臂上验证了该框架的有效性,仅用少量示范即可教授具有不同物理和几何特性的工具完成表面跟随任务。分析表明,机器人通过将预训练策略中对工具-环境接触关系的识别能力迁移到微调策略中,获得了新工具操作技能。此外,融合距离与触觉传感器显著提升了接触状态及环境几何的识别效果。

原文摘要 · Abstract (English)

Tools extend the manipulation abilities of robots, much like they do for humans. Despite human expertise in tool manipulation, teaching robots these skills faces challenges. The complexity arises from the interplay of two simultaneous points of contact: one between the robot and the tool, and another between the tool and the environment. Tactile and proximity sensors play a crucial role in identifying these complex contacts. However, learning tool manipulation using these sensors remains challenging due to limited real-world data and the large sim-to-real gap. To address this, we propose a few-shot tool-use skill transfer framework using multimodal sensing. The framework involves pre-training the base policy to capture contact states common in tool-use skills in simulation and fine-tuning it with human demonstrations collected in the real-world target domain to bridge the domain gap. We validate that this framework enables teaching surface-following tasks using tools with diverse physical and geometric properties with a small number of demonstrations on the Franka Emika robot arm. Our analysis suggests that the robot acquires new tool-use skills by transferring the ability to recognise tool-environment contact relationships from pre-trained to fine-tuned policies. Additionally, combining proximity and tactile sensors enhances the identification of contact states and environmental geometry.

机器人操作少样本学习多模态感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。