用拼图测试大模型协作，发现反馈越细越难合作
AsymPuzl: An Asymmetric Puzzle for multi-agent cooperation
- 设计双模型拼图环境，一方看左半边，另一方看右半边
- 强模型两轮交流就能解题，弱模型常忽略对话或过度纠正
- 简单反馈提升表现，详细联合反馈反而降低成功率
大型语言模型（LLM）代理在多轮、多代理场景中日益受到关注，但现有设置多强调开放式角色扮演，而非可控评估。我们提出AsymPuzl，一个最小但表达力强的双代理拼图环境，旨在隔离信息不对称下的通信行为。每个代理观察符号拼图的互补但不完整视图，需通过消息交换协同求解。使用当前多代及开源LLM进行实验表明：(i) 强模型如GPT-5和Claude-4.0能在两轮内可靠收敛于解法，通过传递完整信息；(ii) 较弱模型常忽略对方消息或过度修正自身假设；(iii) 反馈设计至关重要：简单自反馈可提升成功率，而详细联合反馈反而损害性能。这些发现表明，即使在简单协作任务中，LLM的沟通策略也显著分化，且依赖反馈信号的粒度。AsymPuzl因此成为探测多轮协作极限的测试平台，并为研究协调机制开辟新路径。
原文摘要 · Abstract (English)
Large Language Model (LLM) agents are increasingly studied in multi-turn, multi-agent scenarios, yet most existing setups emphasize open-ended role-play rather than controlled evaluation. We introduce AsymPuzl, a minimal but expressive two-agent puzzle environment designed to isolate communication under information asymmetry. Each agent observes complementary but incomplete views of a symbolic puzzle and must exchange messages to solve it cooperatively. Using a diverse set of current-generation and open-source LLMs, we show that (i) strong models such as GPT-5 and Claude-4.0 reliably converge across puzzle sizes on the solution by sharing complete information in two turns, (ii) weaker models often ignore partner messages or over-correct their hypotheses, and (iii) feedback design is non-trivial: simple self-feedback improves success rates, while detailed joint feedback can hurt performance. These findings show that even in simple cooperative tasks, LLM communication strategies diverge and depend on the granularity of feedback signals. AsymPuzl thus provides a testbed for probing the limits of multi-turn cooperation and opens avenues for studying coordination mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。