从人类示范中学习双手操作的空间约束,提升机器人对物体交互的理解。
Learning Spatial Bimanual Action Models Based on Affordance Regions and Human Demonstrations
- 通过提取物体可操作区域间的空间约束,建模双手动作
- 在模拟任务中验证,实现倒饮料与擀面的精准执行
- 适合研究人机协作与具身智能的开发者参考
本文提出一种新方法,通过分析人类示范中物体可操作区域之间的空间约束(称为可操作约束),学习双手操作动作。可操作区域指物体上可供智能体交互的部分,如瓶底可放置、瓶口可倾倒液体。该方法提取人类示范中可操作约束的变化,构建表征物体交互的空间双手动作模型。为利用该模型,我们建立优化问题,在考虑初始场景、学习到的可操作约束及机器人运动学的前提下,确定多个执行关键点处的最优物体配置。我们在两个示例任务(倒饮料和擀面)的仿真环境中评估该方法,并比较三种可操作约束定义:(i) 笛卡尔空间中可操作区域间的分量距离,(ii) 圆柱空间中的分量距离,(iii) 手工定义的符号化空间可操作约束的满足程度。
原文摘要 · Abstract (English)
In this paper, we present a novel approach for learning bimanual manipulation actions from human demonstration by extracting spatial constraints between affordance regions, termed affordance constraints, of the objects involved. Affordance regions are defined as object parts that provide interaction possibilities to an agent. For example, the bottom of a bottle affords the object to be placed on a surface, while its spout affords the contained liquid to be poured. We propose a novel approach to learn changes of affordance constraints in human demonstration to construct spatial bimanual action models representing object interactions. To exploit the information encoded in these spatial bimanual action models, we formulate an optimization problem to determine optimal object configurations across multiple execution keypoints while taking into account the initial scene, the learned affordance constraints, and the robot's kinematics. We evaluate the approach in simulation with two example tasks (pouring drinks and rolling dough) and compare three different definitions of affordance constraints: (i) component-wise distances between affordance regions in Cartesian space, (ii) component-wise distances between affordance regions in cylindrical space, and (iii) degrees of satisfaction of manually defined symbolic spatial affordance constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。