用信息论分析双手协作，从单段视频生成双臂机器人执行计划。
Information-Theoretic Detection of Bimanual Interactions for Dual-Arm Robot Plan Generation
- 基于香农信息论分析视觉元素间信息流，识别双手协作模式。
- 在自建和公开数据集上实现双臂协同计划生成，效果优于现有方法。
- 适合非专家用户通过视频演示快速编程双臂机器人系统。
示范编程是一种通过人类示范简化机器人编程的方法，但双手任务因手部协调复杂而未被充分探索，也影响了数据采集。本文提出一种单次处理方法，仅需一段双人动作的RGB视频，即可为双臂机器人生成执行计划。为检测手部协调策略，我们应用香农信息论分析场景元素间的动态信息流，并结合场景图特性。生成的计划为模块化行为树，其结构随所需手臂协调方式而变化。我们在多个受试者录制的视频上验证了该框架的有效性，这些视频已开源；同时使用外部公开数据集进行测试。与现有方法相比，该方法在生成双臂系统统一执行计划方面有显著提升。
原文摘要 · Abstract (English)
Programming by demonstration is a strategy to simplify the robot programming process for non-experts via human demonstrations. However, its adoption for bimanual tasks is an underexplored problem due to the complexity of hand coordination, which also hinders data recording. This paper presents a novel one-shot method for processing a single RGB video of a bimanual task demonstration to generate an execution plan for a dual-arm robotic system. To detect hand coordination policies, we apply Shannon's information theory to analyze the information flow between scene elements and leverage scene graph properties. The generated plan is a modular behavior tree that assumes different structures based on the desired arms coordination. We validated the effectiveness of this framework through multiple subject video demonstrations, which we collected and made open-source, and exploiting data from an external, publicly available dataset. Comparisons with existing methods revealed significant improvements in generating a centralized execution plan for coordinating two-arm systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。