让机器人两臂协作预处理物体,提升抓取成功率。
BiPreManip: Learning Affordance-Based Bimanual Preparatory Manipulation through Anticipatory Collaboration
- 基于视觉属性预测最终动作,引导一臂预操作
- 真实与仿真环境下任务成功率显著提升
- 适用于多种物体,泛化能力强,适合复杂操作
日常物品如平板电脑或笔,常难以直接抓握或功能操作(如打开笔帽)。这类任务需要双臂分步、非对称协作:一臂先进行预处理,为另一臂的最终目标动作创造条件。本文提出协同预处理操作,要求理解物体语义与几何特征,预判空间关系,并规划长时程双臂协调动作。为此,我们设计一种基于视觉属性的框架,先构想最终目标动作,再指导一臂执行一系列预操作以促成另一臂后续动作。该属性中心表征支持前瞻性的双臂推理与协作,在不同类别物体间实现良好泛化。大量仿真与真实世界实验表明,相比竞争性基线,本方法在任务成功率和泛化能力上均有显著提升。
原文摘要 · Abstract (English)
Many everyday objects are difficult to directly grasp (e.g., a flat iPad) or manipulate functionally (e.g., opening the cap of a pen lying on a desk). Such tasks require sequential, asymmetric coordination between two arms, where one arm performs preparatory manipulation that enables the other's goal-directed action - for instance, pushing the iPad to the table's edge before picking it up, or lifting the pen body to allow the other hand to remove its cap. In this work, we introduce Collaborative Preparatory Manipulation, a class of bimanual manipulation tasks that demand understanding object semantics and geometry, anticipating spatial relationships, and planning long-horizon coordinated actions between the two arms. To tackle this challenge, we propose a visual affordance-based framework that first envisions the final goal-directed action and then guides one arm to perform a sequence of preparatory manipulations that facilitate the other arm's subsequent operation. This affordance-centric representation enables anticipatory inter-arm reasoning and coordination, generalizing effectively across various objects spanning diverse categories. Extensive experiments in both simulation and the real world demonstrate that our approach substantially improves task success rates and generalization compared to competitive baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。