让不同机器人的智能体自主理解自身物理能力,协同完成复杂任务。
EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agents
- 用机器人自我描述(Robot Resume)替代预设角色,基于URDF和运动学自动生成能力说明。
- 在多楼层物体重组等任务中,系统整体成功率提升40%以上,验证了设计有效性。
- 适合研究异构机器人协作、具身智能与大模型应用的科研人员参考。
异构多机器人系统(HMRS)已成为解决单个机器人无法独立完成的复杂任务的有效方法。当前基于大语言模型的多智能体系统(LLM-based MAS)在软件开发和操作系统等领域表现优异,但在机器人控制中的应用面临独特挑战:每个智能体的能力与其物理构成直接相关,而非预先设定的角色。为此,我们提出一种新型多智能体框架,支持具有不同身体形态和能力的机器人高效协作,并构建新基准Habitat-MAS。核心设计之一是“机器人简历”(Robot Resume):不采用人工预设角色,而是通过智能体自主解析机器人URDF文件并调用运动学工具,生成自身物理能力描述,用于指导任务规划与动作执行。Habitat-MAS基准涵盖操作、感知、导航及跨楼层综合物体重排四类任务,用于评估具身意识推理能力。实验结果表明,机器人简历与分层架构对异构多机器人系统在复杂场景下的有效运行至关重要。
原文摘要 · Abstract (English)
Heterogeneous multi-robot systems (HMRS) have emerged as a powerful approach for tackling complex tasks that single robots cannot manage alone. Current large-language-model-based multi-agent systems (LLM-based MAS) have shown success in areas like software development and operating systems, but applying these systems to robot control presents unique challenges. In particular, the capabilities of each agent in a multi-robot system are inherently tied to the physical composition of the robots, rather than predefined roles. To address this issue, we introduce a novel multi-agent framework designed to enable effective collaboration among heterogeneous robots with varying embodiments and capabilities, along with a new benchmark named Habitat-MAS. One of our key designs is $\textit{Robot Resume}$: Instead of adopting human-designed role play, we propose a self-prompted approach, where agents comprehend robot URDF files and call robot kinematics tools to generate descriptions of their physics capabilities to guide their behavior in task planning and action execution. The Habitat-MAS benchmark is designed to assess how a multi-agent framework handles tasks that require embodiment-aware reasoning, which includes 1) manipulation, 2) perception, 3) navigation, and 4) comprehensive multi-floor object rearrangement. The experimental results indicate that the robot's resume and the hierarchical design of our multi-agent system are essential for the effective operation of the heterogeneous multi-robot system within this intricate problem context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。