验证大模型代理在真实数据上的行为是否符合业务规则。
Formal Verification of Agentic Systems over Operational Data
- 将代理系统建模为带状态的工具调用框架,形式化其执行逻辑。
- 在有限数据域下,验证问题为PSPACE完全,可保证精确性。
- 提出封装器确保数据标识与工具调用同步,适合高可靠性场景。
由大语言模型驱动的代理系统正被部署于持续运营数据的现实工作流中。部署前需验证其是否满足业务规则对流程执行与数据演化的约束。然而,现有方法仅关注代理接口层面的行为限制,无法提供系统级保证。本文研究单一LLM与工具编排框架组成的代理系统,将其形式化为有状态工具使能代理部署(STEAD),定义其语义,并提出基于一阶计算树逻辑(FO-CTL)的验证问题。我们证明该问题是不可判定的;但在有限域下,若数据中不透明标识符的重命名对应工具调用的同步重命名,则可保证FO-CTL规格的精确保留,此时验证为PSPACE完全。我们发现LLM代理可能违反此条件,并提出一个通用封装器,在不破坏已有等变行为的前提下强制满足该条件。我们证明计算该封装所需规范表示是图同构难题。最后,我们在一个案例管理工作的LLM代理实例上展示了该框架的有效性。
原文摘要 · Abstract (English)
Agentic systems driven by large language models (LLMs) are increasingly deployed in real-world workflows where they act on persistent operational data. Before deployment, these systems need to be verified against business requirements that govern workflow execution and data evolution. However, existing approaches do not provide such system-level guarantees, as they mainly constrain or analyse behaviour at the agent's interface level. We study here the verification of agentic systems comprising a single LLM and a tool orchestration harness over relational operational data. We formalise them as Stateful Tool-Enabled Agentic Deployments (STEADs), give their semantics, define the problem of verifying them against First-Order Computation Tree Logic (FO-CTL) specifications, and show that it is undecidable. We identify sufficient conditions for exact preservation of FO-CTL specifications under a finite-domain restriction, over which verification is PSPACE-complete. The key requirement is that renaming opaque identifiers in the data must correspondingly rename the selected tool calls. We show that LLM-driven agents can violate this condition and introduce a canonical deployment wrapper that guarantees it for arbitrary base agents while preserving already-equivariant behaviour. We prove that computing canonical representations required by this construction is graph-isomorphism-hard. Finally, we illustrate our framework on an LLM agent orchestrating a case-management workflow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。