arXiv:2603.18096cs.MAcs.AI2026-03被引 6

为智能体协作AI设计可验证的运行保障框架,解决长期交互中的失效问题。

A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governance

  • 通过消息-动作追踪记录每步执行,用合约约束行为并定位首个违规步骤。
  • 支持压力测试与故障注入,评估系统在异常情况下的容错能力。
  • 将治理机制嵌入运行时,实现对智能体行为的实时控制与限制。

在智能体式AI中,大型语言模型(LLMs)越来越多地用于协调多个智能体及与外部服务、检索组件和共享内存交互。此类系统中的失败不仅限于错误的最终输出,还源于长周期交互、随机决策和外部副作用(如API调用、数据库写入和消息发送)。常见问题包括非终止、角色漂移、不实主张传播,以及通过不可信上下文或外部通道发起的攻击。本文提出一种针对此类系统的保障框架:执行过程被仪器化为带显式步骤与追踪合约的消息-动作追踪(MAT)。合约提供机器可验证结论,定位首个违规步骤,并支持确定性重放。框架包含压力测试,以预算内有界扰动搜索反例;还支持在服务、检索和内存边界进行结构化故障注入,以评估真实运行故障与降级条件下的容错能力。治理作为运行时组件,于语言到动作边界实施每个智能体的能力限制与操作中介(允许、重写、阻断)。为支持跨随机种子、模型和编排配置的比较评估,论文定义了基于追踪的指标:任务成功率、终止可靠性、合约合规率、事实性指标、容错率及治理结果分布。该框架旨在作为通用抽象,支持多智能体LLM系统的测试与评估,促进不同编排设计与配置间的可复现对比。

原文摘要 · Abstract (English)

In Agentic AI, Large Language Models (LLMs) are increasingly used in the orchestration layer to coordinate multiple agents and to interact with external services, retrieval components, and shared memory. In this setting, failures are not limited to incorrect final outputs. They also arise from long-horizon interaction, stochastic decisions, and external side effects (such as API calls, database writes, and message sends). Common failures include non-termination, role drift, propagation of unsupported claims, and attacks via untrusted context or external channels. This paper presents an assurance framework for such Agentic AI systems. Executions are instrumented as Message-Action Traces (MAT) with explicit step and trace contracts. Contracts provide machine-checkable verdicts, localize the first violating step, and support deterministic replay. The framework includes stress testing, formulated as a budgeted counterexample search over bounded perturbations. It also supports structured fault injection at service, retrieval, and memory boundaries to assess containment under realistic operational faults and degraded conditions. Finally, governance is treated as a runtime component, enforcing per-agent capability limits and action mediation (allow, rewrite, block) at the language-to-action boundary. To support comparative evaluations across stochastic seeds, models, and orchestration configurations, the paper defines trace-based metrics for task success, termination reliability, contract compliance, factuality indicators, containment rate, and governance outcome distributions. More broadly, the framework is intended as a common abstraction to support testing and evaluation of multi-agent LLM systems, and to facilitate reproducible comparison across orchestration designs and configurations.

智能体系统AI保障故障注入运行时治理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。