首份大规模研究揭示多智能体AI系统开发现状与痛点
A Large-Scale Study on the Development and Issues of Multi-Agent AI Systems
- 分析8个主流系统4.7k个问题和4.2万次提交，识别三类开发模式
- 40.8%的修改为功能增强，远超修复性维护（27.4%）和适应性更新（24.3%）
- 2023年后问题报告激增，需加强测试、文档与长期维护
多智能体AI系统（MAS），如LangChain、CrewAI和AutoGen，正重塑大语言模型应用的开发与编排方式。然而，这些系统在实际中的演化与维护状况仍不清楚。本文首次对开源MAS进行大规模实证研究，分析了八个领先系统中超过4.2万次唯一提交和4.7千个已解决的问题。分析识别出三种不同的开发模式：持续型、稳定型和爆发型，反映了生态系统成熟度的显著差异。完美性提交占所有变更的40.8%，表明功能增强优先于纠正性维护（27.4%）和适应性更新（24.3%）。问题数据表明，最常见问题是漏洞（22%）、基础设施（14%）和智能体协作挑战（10%）。自2023年起，各框架的问题报告量显著上升。问题平均解决时间从不到一天到约两周不等，分布偏向快速响应，但少数问题需长期关注。这些结果凸显当前生态系统的活力与脆弱性，强调需改进测试基础设施、文档质量与维护实践，以保障长期可靠性与可持续性。
原文摘要 · Abstract (English)
The rapid emergence of multi-agent AI systems (MAS), including LangChain, CrewAI, and AutoGen, has shaped how large language model (LLM) applications are developed and orchestrated. However, little is known about how these systems evolve and are maintained in practice. This paper presents the first large-scale empirical study of open-source MAS, analyzing over 42K unique commits and over 4.7K resolved issues across eight leading systems. Our analysis identifies three distinct development profiles: sustained, steady, and burst-driven. These profiles reflect substantial variation in ecosystem maturity. Perfective commits constitute 40.8% of all changes, suggesting that feature enhancement is prioritized over corrective maintenance (27.4%) and adaptive updates (24.3%). Data about issues shows that the most frequent concerns involve bugs (22%), infrastructure (14%), and agent coordination challenges (10%). Issue reporting also increased sharply across all frameworks starting in 2023. Median resolution times range from under one day to about two weeks, with distributions skewed toward fast responses but a minority of issues requiring extended attention. These results highlight both the momentum and the fragility of the current ecosystem, emphasizing the need for improved testing infrastructure, documentation quality, and maintenance practices to ensure long-term reliability and sustainability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。