通过改进哲学家就餐问题，测试大模型在资源竞争下的协作能力。
DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contention
- 构建可独立调节协议、通信结构和人数的多智能体测试平台。
- 同一模型在不同协议下死锁率从90%降至0%，表明协议决定成败。
- 适合研究AI协作机制或评估多智能体系统鲁棒性的研究人员。
我们提出DPBench，一个用于评估由大型语言模型构建的多智能体系统协调能力的基准。现有基准仅在固定协议下衡量任务成功率；但协调成功或失败的结构性条件尚未被刻画。DPBench将哲学家就餐问题改造为受控实验平台，其中动作协议、通信结构和群体规模可独立变化。我们评估了六种代理：GPT-5.2、Claude Opus 4.5、Grok 4.1、Gemini 2.5 Flash、Llama 4 Maverick 和一个均匀随机基线。在同时行动且N=5时，默认提示下死锁率从GPT-5.2的25.0%（95%威尔逊置信区间[11.2, 46.9]）到Gemini 2.5 Flash的90.0% [74.4, 96.5]不等；顺序行动则被六种中的四种解决。固定模型为Gemini 2.5 Flash时，三个协议变量可使死锁率从90%降至置信区间内的零：三轮预承诺通信（0.0% 对比单轮86.7%）、使用编码经典并发原语的提示（资源排序与对称性打破为0.0%，最小提示为100%），或群体规模从N=5增至N=10（90.0%降至10.0%）。单轮消息传递与历史状态记忆在当前样本量下未改变死锁率。同一模型是否发生死锁，取决于协议而非模型能力。
原文摘要 · Abstract (English)
We present DPBench, a benchmark for evaluating coordination in multi-agent systems built from large language models. Existing benchmarks measure task-level success under a fixed protocol; the structural conditions under which coordination succeeds or fails at all have not been characterised. DPBench adapts the Dining Philosophers problem into a controlled testbed where the action protocol, the communication structure, and the group size each vary independently. We evaluate six agents: GPT-5.2, Claude Opus 4.5, Grok 4.1, Gemini 2.5 Flash, Llama 4 Maverick, and a uniform-random baseline. Under simultaneous action at N=5 with the default prompt, deadlock ranges from 25.0% (95% Wilson CI [11.2, 46.9]) for GPT-5.2 to 90.0% [74.4, 96.5] for Gemini 2.5 Flash; sequential action is solved by four of the six. Holding the model fixed at Gemini 2.5 Flash, three protocol variables drive deadlock from 90% to within CI of zero: three rounds of pre-commitment communication (0.0% vs. single-round 86.7%), a prompt encoding a classical concurrency primitive (0.0% for resource-ordering and symmetry-breaking, against 100% for the minimal prompt), or doubling the group from N=5 to N=10 (90.0% to 10.0%). Single-round messaging and memory of past timesteps do not change the rate at the sample size we ran. Whether the same model coordinates or deadlocks is determined by the protocol, not by the model's capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。