提出组合约束框架，自动构建多智能体物理协作训练数据集
RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints
- 用组合约束设计接口，实现智能体与物理世界的无缝交互
- 构建首个多智能体操作基准RoboFactory，支持不同难度任务评估
- 验证模仿学习在复杂协作中的有效性，指导安全高效系统设计
设计高效的具身多智能体系统对于解决跨领域的复杂现实任务至关重要。由于具身多智能体系统的复杂性，现有方法无法自动生成安全且高效的训练数据。为此，我们提出了具身多智能体系统的组合约束概念，解决了智能体间协作带来的挑战。我们设计了多种针对不同类型约束的接口，实现与物理世界的无缝交互。基于组合约束和定制接口，我们开发了一个具身多智能体系统的自动化数据采集框架，并引入首个具身多智能体操作基准RoboFactory。基于该基准，我们适配并评估了模仿学习方法在不同难度智能体任务中的表现。此外，我们探索了多智能体模仿学习的架构与训练策略，旨在构建安全高效的具身多智能体系统。
原文摘要 · Abstract (English)
Designing effective embodied multi-agent systems is critical for solving complex real-world tasks across domains. Due to the complexity of multi-agent embodied systems, existing methods fail to automatically generate safe and efficient training data for such systems. To this end, we propose the concept of compositional constraints for embodied multi-agent systems, addressing the challenges arising from collaboration among embodied agents. We design various interfaces tailored to different types of constraints, enabling seamless interaction with the physical world. Leveraging compositional constraints and specifically designed interfaces, we develop an automated data collection framework for embodied multi-agent systems and introduce the first benchmark for embodied multi-agent manipulation, RoboFactory. Based on RoboFactory benchmark, we adapt and evaluate the method of imitation learning and analyzed its performance in different difficulty agent tasks. Furthermore, we explore the architectures and training strategies for multi-agent imitation learning, aiming to build safe and efficient embodied multi-agent systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。