用图结构语言评估智能体行为确定性，确保推理路径可复现。
EDGE: Engine for Deterministic Graph Evaluation through Conversation Simulation from Graph Structured DSL Configuration
- 基于图结构的领域专用语言构建智能体推理模型
- 通过遍历对话路径捕捉完整行为空间，验证输出一致性
- 新指标量化响应、轨迹与语义一致性，适合复杂工作流评估
随着智能体系统演变为复杂的多智能体编排流程，亟需系统化框架来衡量智能体行为的一致性与确定性。本文提出一种基于AgentGraph的形式化评估方法，该方法由领域专用语言驱动，通过动态可调的有向图表示智能体推理过程。利用图遍历算法，全面枚举对话路径，生成涵盖智能体全部行为空间的评估集，并系统重放这些可复现的轨迹，对比实际输出与状态转移是否符合预期的DSL规范。为量化可靠性，我们定义了新型度量指标，用于评估响应与轨迹确定性、结构符合度及语义一致性，覆盖精确重播及其语言变体。实验结果表明，使用AgentGraph或LangGraph等框架配置节点转换的智能体，相比未采用受控转换的智能体，展现出显著更高的确定性。
原文摘要 · Abstract (English)
As agentic systems evolve into complex multi agent orchestration workflows, there is a growing and critical need for systematic frameworks that measures an agent's behavioral consistency and determinism. In this paper, we introduce a formal evaluation methodology that is grounded in AgentGraph, a planner powered by a domain specific language that represents agent reasoning through a dynamically adjustable directed graph. We leverage this structural formalism and utilize graph traversal algorithms that exhaustively enumerate conversational paths, forming a comprehensive evaluation set that captures the agent's complete behavioral space. We then systematically replay these reproducible trajectories to compare observed outputs and state transitions against the intended DSL specification. To quantify reliability, we define novel metrics that measure response and trajectory determinism, structural adherence and semantic consistency across both exact replays and their linguistic variants. Our system's results demonstrate that agents configured using frameworks like AgentGraph and LangGraph with explicitly structured node transitions show superior determinism over agents that are not configured with controlled transitions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。