用大模型让不同机器人自主协作完成复杂任务
MHRC: Closed-loop Decentralized Multi-Heterogeneous Robot Collaboration with Large Language Models
- 通过自然语言和思维链提示实现多类型机器人的去中心化协同
- 在三种房间布局中成功完成探索、搬运、整理等任务
- 适合研究人机协作与智能机器人系统的开发者
大语言模型（LLMs）与机器人结合显著提升了机器人在感知、认知和任务规划方面的能力。自然语言接口为表达异构机器人的能力差异提供了统一方式，促进了机器人间的通信，并实现了无缝的任务分配与协作。目前，利用大模型实现去中心化的多异构机器人协同任务仍是一个未充分探索的研究方向。本文提出一种新框架，利用大模型实现多种异构机器人之间的去中心化协作。框架支持移动机器人、操作机器人及移动操作机器人三类，共同完成探索、运输和组织等任务。我们设计了丰富的文本反馈机制和思维链（CoT）提示，以提升任务规划效率与系统整体性能。移动操作机器人可灵活调整基座位置，确保抓取条件最优；操作机器人能理解任务需求，在必要时寻求协助并合理处理物体；移动机器人则广泛探索环境，绘制物体位置图，并将信息传递给移动操作机器人，从而提高任务执行效率。我们在PyBullet中进行了评估，设置了三种不同的房间布局和三种操作任务，测试了多种大模型并开展了消融实验以评估各模块贡献。实验结果验证了所提框架的有效性与必要性。
原文摘要 · Abstract (English)
The integration of large language models (LLMs) with robotics has significantly advanced robots' abilities in perception, cognition, and task planning. The use of natural language interfaces offers a unified approach for expressing the capability differences of heterogeneous robots, facilitating communication between them, and enabling seamless task allocation and collaboration. Currently, the utilization of LLMs to achieve decentralized multi-heterogeneous robot collaborative tasks remains an under-explored area of research. In this paper, we introduce a novel framework that utilizes LLMs to achieve decentralized collaboration among multiple heterogeneous robots. Our framework supports three robot categories, mobile robots, manipulation robots, and mobile manipulation robots, working together to complete tasks such as exploration, transportation, and organization. We developed a rich set of textual feedback mechanisms and chain-of-thought (CoT) prompts to enhance task planning efficiency and overall system performance. The mobile manipulation robot can adjust its base position flexibly, ensuring optimal conditions for grasping tasks. The manipulation robot can comprehend task requirements, seek assistance when necessary, and handle objects appropriately. Meanwhile, the mobile robot can explore the environment extensively, map object locations, and communicate this information to the mobile manipulation robot, thus improving task execution efficiency. We evaluated the framework using PyBullet, creating scenarios with three different room layouts and three distinct operational tasks. We tested various LLM models and conducted ablation studies to assess the contributions of different modules. The experimental results confirm the effectiveness and necessity of our proposed framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。