用大模型+多智能体规划,让双臂机器人自动完成复杂长时序任务。
LLM+MAP: Bimanual Robot Task Planning using Large Language Models and Planning Domain Definition Language
- 结合大模型推理与多智能体规划,实现双臂任务的自动分解与分配。
- 在模拟任务中成功率提升至92%,计划步数减少41%。
- 适合需要长时序规划的双臂机器人研究者使用。
双臂机器人操作具有显著灵活性,但因双手在空间与时间上的协调复杂性而面临挑战。现有研究多聚焦于达到人类级操作技能,却忽视了长时间尺度的任务规划。凭借出色的上下文学习和零样本生成能力,大语言模型(LLMs)已被应用于多种机器人实体以促进任务规划。然而,LLMs在长时序推理中仍存在错误和幻觉问题,生成计划缺乏逻辑正确性保证。此前工作如LLM+P虽将符号规划器与LLMs结合,但尚未成功应用于双臂机器人。双臂操作带来新挑战,不仅需有效任务分解,还需高效任务分配。为此,本文提出LLM+MAP框架,融合大模型推理与多智能体规划,实现双臂任务的自动化、高效规划。实验基于GPT-4o作为后端,在多种复杂长时序操作任务上进行模拟验证。与直接由LLMs生成的计划(包括GPT-4o、V3、o1和R1)相比,通过规划时间、成功率、组扣分及计划步数减少率等指标评估,证实了LLM+MAP的优越性能,并为机器人推理提供了深入洞察。代码已公开于https://github.com/Kchu/LLM-MAP。
原文摘要 · Abstract (English)
Bimanual robotic manipulation provides significant versatility, but also presents an inherent challenge due to the complexity involved in the spatial and temporal coordination between two hands. Existing works predominantly focus on attaining human-level manipulation skills for robotic hands, yet little attention has been paid to task planning on long-horizon timescales. With their outstanding in-context learning and zero-shot generation abilities, Large Language Models (LLMs) have been applied and grounded in diverse robotic embodiments to facilitate task planning. However, LLMs still suffer from errors in long-horizon reasoning and from hallucinations in complex robotic tasks, lacking a guarantee of logical correctness when generating the plan. Previous works, such as LLM+P, extended LLMs with symbolic planners. However, none have been successfully applied to bimanual robots. New challenges inevitably arise in bimanual manipulation, necessitating not only effective task decomposition but also efficient task allocation. To address these challenges, this paper introduces LLM+MAP, a bimanual planning framework that integrates LLM reasoning and multi-agent planning, automating effective and efficient bimanual task planning. We conduct simulated experiments on various long-horizon manipulation tasks of differing complexity. Our method is built using GPT-4o as the backend, and we compare its performance against plans generated directly by LLMs, including GPT-4o, V3 and also recent strong reasoning models o1 and R1. By analyzing metrics such as planning time, success rate, group debits, and planning-step reduction rate, we demonstrate the superior performance of LLM+MAP, while also providing insights into robotic reasoning. Code is available at https://github.com/Kchu/LLM-MAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。