首次实证研究开发者在大模型智能体框架中的实践，揭示框架设计的关键差距。
An Empirical Study of Agent Developer Practices in AI Agent Frameworks
- 通过分析1.19万条开发讨论，对比十类框架在五维度表现
- 超80%开发者难选适配自身需求的框架，学习成本与可维护性成主要痛点
- 为未来框架设计提供实证依据，适合框架研发者与高级开发者参考
大语言模型（LLMs）的兴起推动了智能体的快速发展，催生出众多智能体框架。这些框架作为软件工具包和库，提供标准化组件、抽象接口和编排机制，旨在简化智能体开发。尽管广泛使用，但其实际应用情况及对开发流程的影响仍缺乏深入探索。不同框架在使用中面临相似问题，表明这些问题值得重视，并需进一步优化框架设计。同时，随着框架数量持续增长，超过80%的开发者报告难以找到符合特定开发需求的框架。本文开展首个基于LLM的智能体框架实证研究，通过收集并分析十个已识别框架的11,910条开发者讨论，从开发效率、功能抽象、学习成本、性能优化和可维护性五个维度进行比较。结果揭示各框架在满足开发者需求方面存在显著差异。本研究为大模型驱动的智能体框架生态系统提供了系列发现与启示，为未来框架设计及开发者实践提供重要参考。
原文摘要 · Abstract (English)
The rise of large language models (LLMs) has sparked a surge of interest in agents, leading to the rapid growth of agent frameworks. Agent frameworks are software toolkits and libraries that provide standardized components, abstractions, and orchestration mechanisms to simplify agent development. Despite widespread use of agent frameworks, their practical applications and how they influence the agent development process remain underexplored. Different agent frameworks encounter similar problems during use, indicating that these recurring issues deserve greater attention and call for further improvements in agent framework design. Meanwhile, as the number of agent frameworks continues to grow and evolve, more than 80% of developers report difficulties in identifying the frameworks that best meet their specific development requirements. In this paper, we conduct the first empirical study of LLM-based agent frameworks, exploring real-world experiences of developers in building AI agents. To compare how well the agent frameworks meet developer needs, we further collect developer discussions for the ten previously identified agent frameworks, resulting in a total of 11,910 discussions. Finally, by analyzing these discussions, we compare the frameworks across five dimensions: development efficiency, functional abstraction, learning cost, performance optimization, and maintainability, which refers to how easily developers can update and extend both the framework itself and the agents built upon it over time. Our comparative analysis reveals significant differences among frameworks in how they meet the needs of agent developers. Overall, we provide a set of findings and implications for the LLM-driven AI agent framework ecosystem and offer insights for the design of future LLM-based agent frameworks and agent developers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。