为多智能体协作规划设计高效时间评估基准
ParaCook: On Time-Efficient Planning for Multi-Agent Systems
- 基于烹饪任务模拟多智能体并行协作，简化动作空间聚焦策略规划
- 现有大模型规划效率低，难以协调并行操作，平均完成时间高出理想值37%
- 适合研究多智能体协同、时间敏感任务的算法开发者
大型语言模型在长时序现实任务规划中表现出强大的推理能力，但现有智能体基准主要关注任务完成度，忽视了并行与异步操作中的时间效率。为此，我们提出ParaCook，一个面向时间高效协作规划的基准。受Overcooked游戏启发，ParaCook构建了一个包含多种挑战性交互的多智能体系统环境，以烹饪任务为实例，采用简化动作空间，突出战略并行规划的核心挑战。对当前先进大模型的全面评估显示，现有方法生成的计划存在次优问题，难以有效处理并行动作或协调。分析还揭示大模型在抽象任务中具备高阶并行优化潜力。ParaCook提供可扩展的评估框架，支持复杂度调节，为开发和评测时间效率感知的多智能体规划奠定基础。代码与数据已开源于https://github.com/zsq259/ParaCook。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit strong reasoning abilities for planning long-horizon, real-world tasks, yet existing agent benchmarks focus on task completion while neglecting time efficiency in parallel and asynchronous operations. To address this, we present ParaCook, a benchmark for time-efficient collaborative planning. Inspired by the Overcooked game, ParaCook provides an environment for various challenging interaction planning of multi-agent systems that are instantiated as cooking tasks, with a simplified action space to isolate the core challenge of strategic parallel planning. Through a comprehensive evaluation of state-of-the-art LLMs, we find that current approaches achieve suboptimal plans, which struggle with parallel actions or coordination. Our analysis also reveals LLMs' potential on abstract tasks where they can focus on high-level parallel optimization. ParaCook provides a scalable evaluation framework with adjustable complexity, establishing a foundation for developing and assessing time efficiency-aware multi-agent planning. The code and data are available at https://github.com/zsq259/ParaCook.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。