研究多个AI程序员如何协作写斐波那契数列,发现不同模型组合表现差异巨大。
Understanding Conversational Patterns in Multi-agent Programming: A Case Study on Fibonacci Game Development

- 让两个AI角色(设计者和程序员)用12种模型组合对话编程
- 唯一成功的是DeepSeek-R1配对,首轮就出正确代码且稳定保持
- 多数组合中途跑偏或反复争吵,说明协作机制需深入研究
大型语言模型在软件工程中的应用日益广泛,但其在自主、角色导向的多智能体协作方面的潜力仍待探索。理解多个基于LLM的智能体如何协调、保持角色一致并收敛到解决方案,对软件工程至关重要,因为随意让智能体交互并不能保证得出正确或稳定的成果。近期实证研究显示,缺乏结构或理解不当的交互动态可能导致错误传播、过早达成错误共识或长期分歧而无法收敛,即使早期已出现正确部分解。作为解决这一未充分研究领域的初步步骤,我们系统分析了12种模型组合(来自7个开源LLM:Gemma 2、Gemma 3、LLaMA 3.2、LLaMA 3.3、DeepSeek-R1、MiniCPM、Qwen3)中两个智能体(设计师与程序员)之间的对话。我们的系统方法揭示了多智能体交互的三个关键维度:效率(收敛速度与稳定性)、一致性(通过BLEU和ROUGE衡量的角色对齐程度)以及有效性(编译成功率与错误修复程度)。结果显示,DeepSeek-R1:DeepSeek-R1组合是唯一从第一轮就收敛到正确解并持续保持到最终迭代的组合;而LLaMA 3.2:LLaMA 3.2与Qwen3:Qwen3虽偏离正确解,但表现出强角色对齐;其余组合均偏离任务,未能收敛。这些发现推进了对代理编程的理解,并强调了未来需进一步研究收敛与终止条件的校准。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly applied to software engineering (SE), yet their potential for autonomous, role-oriented collaboration remains largely underexplored. Understanding how multiple LLM-based agents coordinate, maintain role alignment, and converge on solutions is critical for SE, as naively allowing agents to interact does not reliably lead to correct or stable outcomes. Recent empirical studies show that unstructured or poorly understood interaction dynamics can result in error propagation, premature consensus on incorrect solutions, or prolonged disagreement that prevents convergence, even when correct partial solutions are present early in the interaction. As an initial step towards addressing this underexplored area, we undertake a systematic analysis of conversations between two agents, a Designer and a Programmer across 12 model combinations from 7 open-source LLMs (Gemma 2, Gemma 3, LLaMA 3.2, LLaMA 3.3, DeepSeek-R1, MiniCPM, and Qwen3). Our systematic approach reveals three key dimensions of multi-agent interaction: efficiency (the speed and stability of convergence), consistency (the degree of role alignment visualized by BLEU and ROUGE), and effectiveness (the extent of compilation success and error resolution). Results show that the DeepSeek-R1:DeepSeek-R1 pair was unique in converging to the correct solution from the very first iteration and sustaining it consistently to the final iteration, while LLaMA 3.2:LLaMA 3.2 and Qwen3:Qwen3 demonstrated strong Designer:Programmer role alignment despite of diverging from the correct solution. The other pairs deviated from the task, never to converge to a result. These findings advance understanding of agentic programming and highlight the need for further research on understanding and calibrating convergence and stop conditions essential for future autonomous SE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。