arXiv:2501.15071cs.RO2025-01被引 5

用人类注视点分解机器人操作任务,提升技能复用性。

Gaze-Guided Task Decomposition for Imitation Learning in Robotic Manipulation

  • 基于人类注视转移实现动作任务自动分段
  • 在多任务演示中保持分段一致性,提升学习效率
  • 无需额外标注,适合各类机器人系统部署

在机器人操作的模仿学习中,将任务分解为子任务可促进已学技能的复用与组合,而非简单复制示范动作。人类在操作物体时,目光与手部动作密切相关。本文假设:模仿者通过注视特定关键点并转移视线,能自然划分出操作子任务。提出一种基于注视转移的简单且鲁棒的任务分解方法。利用遥操作采集示范数据时,记录人类操作者的注视行为作为替代,实现对所有示范的一致性分段。该方法在多种任务演示上进行了评估,验证了生成子任务的合理性与一致性。在不同超参数设置下的广泛测试进一步证明其稳健性,适用于多样化的机器人系统。代码已开源:https://github.com/crumbyRobotics/GazeTaskDecomp。

原文摘要 · Abstract (English)

In imitation learning for robotic manipulation, decomposing object manipulation tasks into sub-tasks enables the reuse of learned skills and the combination of learned behaviors to perform novel tasks, rather than simply replicating demonstrated motions. Human gaze is closely linked to hand movements during object manipulation. We hypothesize that an imitating agent's gaze control, fixating on specific landmarks and transitioning between them, simultaneously segments demonstrated manipulations into sub-tasks. This study proposes a simple yet robust task decomposition method based on gaze transitions. Using teleoperation, a common modality in robotic manipulation for collecting demonstrations, in which a human operator's gaze is measured and used for task decomposition as a substitute for an imitating agent's gaze. Our approach ensures consistent task decomposition across all demonstrations for each task, which is desirable in contexts such as machine learning. We evaluated the method across demonstrations of various tasks, assessing the characteristics and consistency of the resulting sub-tasks. Furthermore, extensive testing across different hyperparameter settings confirmed its robustness, making it adaptable to diverse robotic systems. Our code is available at https://github.com/crumbyRobotics/GazeTaskDecomp.

模仿学习任务分解视觉注意机器人操作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。