测试大模型在受限条件下的群体协作能力，发现其难以应对长期规划和动态策略挑战。
Benchmarking LLMs' Swarm intelligence
- 设计5个基础任务，在2D网格中让大模型作为分布式智能体仅依赖局部感知与通信
- 零样本测试显示主流模型（如deepseek-v3）在任务间表现差异大，缺乏稳健长程规划能力
- 开源工具包支持可复现研究，适合关注分布式智能与群体行为的学者
大型语言模型（LLMs）展现出复杂推理潜力，但其在多智能体系统（MAS）中以类群集方式运行时——即受限于局部感知与通信——的协同能力仍鲜有探索。现有基准未能充分捕捉智能体在时空信息不完整情况下的去中心化协调难题。为此，我们提出SwarmBench，一个新型基准，用于系统评估LLMs作为去中心化智能体的群集智能能力。SwarmBench包含五个基础多智能体协调任务（追捕、同步、觅食、聚集、运输），在可配置的二维网格环境中运行，强制智能体仅依赖局部感官输入（k×k视野）和局部通信。我们提出协调有效性度量并分析涌现的群体动态。对领先模型（如deepseek-v3、o4-mini）的零样本评估揭示显著的任务依赖性性能差异。尽管观察到一些基础协同行为，结果表明当前LLMs在不确定性环境下仍严重缺乏稳健的长距离规划与自适应策略形成能力。在类群集约束下评估LLMs对于理解其在未来的去中心化智能系统中的应用价值至关重要。我们发布SwarmBench为开放可扩展工具包——基于可定制物理系统——提供环境、提示、评估脚本及完整数据集。该工具旨在推动基于LLM的多智能体系统协调的可复现研究，以及严重信息去中心化下涌现集体行为的理论探索。代码库地址：https://github.com/x66ccff/swarmbench。
原文摘要 · Abstract (English)
Large Language Models (LLMs) show potential for complex reasoning, yet their capacity for emergent coordination in Multi-Agent Systems (MAS) when operating under strict swarm-like constraints-limited local perception and communication-remains largely unexplored. Existing benchmarks often do not fully capture the unique challenges of decentralized coordination when agents operate with incomplete spatio-temporal information. To bridge this gap, we introduce SwarmBench, a novel benchmark designed to systematically evaluate the swarm intelligence capabilities of LLMs acting as decentralized agents. SwarmBench features five foundational MAS coordination tasks (Pursuit, Synchronization, Foraging, Flocking, Transport) within a configurable 2D grid environment, forcing agents to rely solely on local sensory input ($k\times k$ view) and local communication. We propose metrics for coordination effectiveness and analyze emergent group dynamics. Zero-shot evaluations of leading LLMs (e.g., deepseek-v3, o4-mini) reveal significant task-dependent performance variations. While some rudimentary coordination is observed, our results indicate that current LLMs significantly struggle with robust long-range planning and adaptive strategy formation under the uncertainty inherent in these decentralized scenarios. Assessing LLMs under such swarm-like constraints is crucial for understanding their utility in future decentralized intelligent systems. We release SwarmBench as an open, extensible toolkit-built on a customizable physical system-providing environments, prompts, evaluation scripts, and comprehensive datasets. This aims to foster reproducible research into LLM-based MAS coordination and the theoretical underpinnings of emergent collective behavior under severe informational decentralization. Our code repository is available at https://github.com/x66ccff/swarmbench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。