arXiv:2411.17636cs.ROcs.AI2024-11被引 27

用多个AI代理协作,让机器人零样本学会复杂操作。

MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation

  • 分角色协作:高阶规划与低阶控制由不同AI代理负责
  • 每步后反馈环境信息,能动态重规划应对失败
  • 无需预训练技能或示例,新任务直接上手

大型语言模型(LLMs)在机器人操作和导航等任务中展现出强大的规划能力。尽管近期研究已将LLM用于高阶和低阶规划,但这些方法常面临长周期任务中的幻觉问题,且因单次生成计划而缺乏实时反馈,适应性差。为此,我们提出一种新型多代理LLM框架——MALMM,将高阶规划与低阶控制代码生成分配给专用的LLM代理,并由一个额外代理动态管理状态转换。通过每一步后获取环境观测,该框架可有效处理中间失败并实现自适应重规划。不同于现有方法,本方案不依赖预训练技能策略或上下文学习示例,可在九个RLBench任务中实现零样本泛化,涵盖长周期任务,显著克服了当前基于LLM的机器人操作方法的核心局限。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotics manipulation and navigation. While recent efforts in robotics have leveraged LLMs both for high-level and low-level planning, these approaches often face significant challenges, such as hallucinations in long-horizon tasks and limited adaptability due to the generation of plans in a single pass without real-time feedback. To address these limitations, we propose a novel multi-agent LLM framework, Multi-Agent Large Language Model for Manipulation (MALMM) that distributes high-level planning and low-level control code generation across specialized LLM agents, supervised by an additional agent that dynamically manages transitions. By incorporating observations from the environment after each step, our framework effectively handles intermediate failures and enables adaptive re-planning. Unlike existing methods, our approach does not rely on pre-trained skill policies or in-context learning examples and generalizes to a variety of new tasks. We evaluate our approach on nine RLBench tasks, including long-horizon tasks, and demonstrate its ability to solve robotics manipulation in a zero-shot setting, thereby overcoming key limitations of existing LLM-based manipulation methods.

机器人操作多智能体零样本大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。