提出新基准Tool-RoCo，评估大模型在多机器人协作中的自主组织能力。
Tool-RoCo: An Agent-as-Tool Self-organization Large Language Model Benchmark in Multi-robot Cooperation
- 将其他智能体视为工具，通过工具调用评估协作与自组织能力。
- 仅7.09%工具调用用于合作，96.42%为激活工具，显示模型不善动态调整。
- 适合研究多智能体自主性、协作机制的学者与开发者参考。
本研究提出Tool-RoCo，一个基于RoCo多机器人协作基准的新评估框架，用于衡量大语言模型（LLMs）在长期多智能体协作中的表现。现有基于LLM的多智能体系统依赖预设编排，忽视了智能体的自主性。Tool-RoCo将其他智能体视为工具，引入协作工具，利用工具调用评估协作与自组织能力。每个智能体（LLM）根据当前状态从候选工具集中选择并接收反馈，后续迭代调整。设计四种LLM范式：（1）集中式协作，单一LLM分配工具；（2）集中式自组织，中央LLM自主激活/停用智能体；（3）去中心化协作，各智能体使用本地信息调用工具；（4）自组织，随机初始智能体发起协作，通过工具调用激活其他智能体。包含三个任务：SORT、PACK和CABINET，以评估格式与参数准确率及智能体协调性。实验结果表明，协作工具占比仅7.09%，激活工具占比达96.42%，说明当前LLM很少主动调用其他智能体作为助手，也极少停用活跃智能体进行适应性协调。Tool-RoCo为评估多智能体任务中LLM的自主性与协作能力提供了系统性基准。代码与演示：https://github.com/ColaZhang22/Tool-Roco
原文摘要 · Abstract (English)
This study proposes Tool-RoCo, a novel benchmark for evaluating large language models (LLMs) in long-term multi-agent cooperation based on RoCo, a multi-robot cooperative benchmark. Recent research on LLM-based multi-agent systems has relied on predefined orchestration, while ignoring agent autonomy. Tool-RoCo treats other agents as tools and introduces cooperative tools, leveraging tool usage to evaluate multi-agent cooperation and self-organization. Tool usage means that each agent (LLM) selects a tool from a candidate set based on the current state, receives feedback, and adjusts its selection in subsequent rounds. To evaluate different autonomy levels, we propose four LLM paradigms: (1) centralized cooperation, where a single LLM allocates tools to all agents; (2) centralized self-organization, where a central LLM autonomously activates agents while keeping others inactive; (3) decentralized cooperation, where each agent has its own LLM and calls tools based on local information; and (4) self-organization, where a randomly chosen initial agent can request collaboration, activating additional agents via tool calls. Tool-RoCo includes three multi-robot tasks, SORT, PACK, and CABINET, to measure format and parameter accuracy and agent coordination through tool usage. The results using several LLMs showed that cooperative tools accounted for only 7.09% of all tools, indicating that LLM-based agents rarely invoked others as assistants. Moreover, activation tools accounted for 96.42%, suggesting that current LLMs tend to maintain active agents while seldom deactivating them for adaptive coordination. Tool-RoCo provides a systematic benchmark to evaluate LLM autonomy and cooperation in multi-agent tasks. Code and Demo: https://github.com/ColaZhang22/Tool-Roco
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。