提出探索与聚焦操作新问题,构建基准并设计双臂主动感知策略。
Towards Exploratory and Focused Manipulation with Bimanual Active Perception: A New Problem, Benchmark and Strategy
- 定义探索与聚焦操作(EFM)新问题,强调主动获取任务所需信息。
- 建立包含10个任务的EFM-10基准,涵盖4类需探索或专注的操作场景。
- 提出双臂主动感知(BAP)策略,实测在模仿学习中显著提升性能。
近期,由于机器人头部安装主摄像头导致视觉遮挡更频繁,主动视觉重新成为操作领域的重要概念。本文反思视觉遮挡本质在于缺乏任务完成所需信息,由此提出更根本的问题:探索与聚焦操作(Exploratory and Focused Manipulation, EFM)。该问题关注通过主动采集信息来完成需要探索或专注的复杂操作任务。为初步应对此问题,我们构建了包含10个任务的EFM-10基准,分为4类,符合上述定义。同时提出双臂主动感知(Bimanual Active Perception, BAP)策略,利用一臂提供主动视觉,另一臂提供力觉传感以协同操作。基于此,我们收集了名为BAPData的数据集用于这些任务。通过模仿学习方法,我们验证了BAP策略的有效性。我们希望EFM-10基准与BAP策略能成为该方向研究的基石。项目网站:EFManipulation.github.io。
原文摘要 · Abstract (English)
Recently, active vision has reemerged as an important concept for manipulation, since visual occlusion occurs more frequently when main cameras are mounted on the robot heads. We reflect on the visual occlusion issue and identify its essence as the absence of information useful for task completion. Inspired by this, we come up with the more fundamental problem of Exploratory and Focused Manipulation (EFM). The proposed problem is about actively collecting information to complete challenging manipulation tasks that require exploration or focus. As an initial attempt to address this problem, we establish the EFM-10 benchmark that consists of 4 categories of tasks that align with our definition (10 tasks in total). We further come up with a Bimanual Active Perception (BAP) strategy, which leverages one arm to provide active vision and another arm to provide force sensing while manipulating. Based on this idea, we collect a dataset named BAPData for the tasks in EFM-10. With the dataset, we successfully verify the effectiveness of the BAP strategy in an imitation learning manner. We hope that the EFM-10 benchmark along with the BAP strategy can become a cornerstone that facilitates future research towards this direction. Project website: EFManipulation.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。