让机器人同时学会动作顺序与时间安排,实现双手协同操作。
Unified Learning of Temporal Task Structure and Action Timing for Bimanual Robot Manipulation
- 用三维高斯混合模型表示动作间的时间关系,捕捉细节时序特征。
- 通过DPLL算法找出所有不冲突的动作顺序组合,支持多种执行模式。
- 结合符号与非符号约束生成可执行的带时间参数的机器人计划,适合复杂双手任务。
时间任务结构对双手操作至关重要:机器人不仅需知道一个动作何时先于或重叠于另一个,还需明确每个动作的起止时间及持续时长。符号化时间关系支持高层任务推理和多种执行路径,而具体时间参数则对双臂协调执行不可或缺。现有方法分别处理这两层,导致高层规划与底层运动同步之间存在断层。本文提出一种从人类示范中联合学习符号与非符号时间约束的方法,并生成可执行的带时间参数的任务计划。贡献包括:(i) 基于多变量高斯混合模型的三维时间表示,用于建模动作间的子符号时间关系;(ii) 基于戴维斯-普特南-洛贝兰-洛文德尔(DPLL)算法的方法,识别并排序所有无矛盾的阿伦关系分配,对应任务的不同模式;(iii) 一种基于优化的规划系统,融合已识别的符号与非符号时间约束,生成用于机器人执行的时序参数化计划。我们在多个数据集上评估该方法,结果表明其生成的计划在时间上更接近人类示范,优于最典型示范基线。
原文摘要 · Abstract (English)
Temporal task structure is fundamental for bimanual manipulation: a robot must not only know that one action precedes or overlaps another, but also when each action should occur and how long it should take. While symbolic temporal relations enable high-level reasoning about task structure and alternative execution sequences, concrete timing parameters are equally essential for coordinating two hands at the execution level. Existing approaches address these two levels in isolation, leaving a gap between high-level task planning and low-level movement synchronization. This work presents an approach for learning both symbolic and subsymbolic temporal task constraints from human demonstrations and deriving executable, temporally parametrized plans for bimanual manipulation. Our contributions are (i) a 3-dimensional representation of timings between two actions with methods based on multivariate Gaussian Mixture Models to represent temporal relationships between actions on a subsymbolic level, (ii) a method based on the Davis-Putnam-Logemann-Loveland (DPLL) algorithm that finds and ranks all contradiction-free assignments of Allen relations to action pairs, representing different modes of a task, and (iii) an optimization-based planning system that combines the identified symbolic and subsymbolic temporal task constraints to derive temporally parametrized plans for robot execution. We evaluate our approach on several datasets, demonstrating that our method generates temporally parametrized plans closer to human demonstrations than the most characteristic demonstration baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。