统一AI代理定义与评估标准,让不同框架的智能体可互操作。
Open Agent Specification (Agent Spec): A Unified Representation for AI Agents
- 用声明式语言定义智能体组件与流程,跨框架通用
- 在4个运行时、3个基准上实现一致评估,性能可比
- 提供工具包支持主流框架,适合开发与研究者使用
智能体框架的泛滥导致其定义、执行和评估方式碎片化。现有系统在抽象层级、数据流语义和工具集成上差异显著,难以共享或复现工作流。本文提出开放智能体规范(Agent Spec),一种跨框架兼容的声明式语言,统一定义AI智能体及智能体工作流,提升可重用性、可移植性和互操作性。Agent Spec定义了通用组件、控制与数据流语义及数据结构,使一个智能体可一次定义、多运行时执行。同时引入标准化评估工具集,类似HELM对大模型的评估,实现跨框架的性能、鲁棒性与效率一致性比较。我们在四个不同运行时(LangGraph、CrewAI、AutoGen、WayFlow)上,基于三个基准(SimpleQA Verified、τ²-Bench、BIRD-SQL)进行了验证。配套提供Python SDK(PyAgentSpec)、参考运行时(WayFlow)及主流框架适配器。该规范弥合了以模型为中心与以智能体为中心的标准与评估鸿沟,为可靠、可重用、可移植的智能体系统奠定基础。
原文摘要 · Abstract (English)
The proliferation of agent frameworks has led to fragmentation in how agents are defined, executed, and evaluated. Existing systems differ in their abstractions, data flow semantics, and tool integrations, making it difficult to share or reproduce workflows. We introduce Open Agent Specification (Agent Spec), a declarative language that defines AI agents and agentic workflows in a way that is compatible across frameworks, promoting reusability, portability and interoperability of AI agents. Agent Spec defines a common set of components, control and data flow semantics, and schemas that allow an agent to be defined once and executed across different runtimes. Agent Spec also introduces a standardized Evaluation harness to assess agent behavior and agentic workflows across runtimes - analogous to how HELM and related harnesses standardized LLM evaluation - so that performance, robustness, and efficiency can be compared consistently across frameworks. We demonstrate this using four distinct runtimes (LangGraph, CrewAI, AutoGen, and WayFlow) evaluated over three different benchmarks (SimpleQA Verified, $τ^2$-Bench and BIRD-SQL). We provide accompanying toolsets: a Python SDK (PyAgentSpec), a reference runtime (WayFlow), and adapters for popular frameworks (e.g., LangGraph, AutoGen, CrewAI). Agent Spec bridges the gap between model-centric and agent-centric standardization & evaluation, laying the groundwork for reliable, reusable, and portable agentic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。