将自然语言工作流编译为可执行的控制流程图,提升大模型代理的准确执行能力。
COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution
- 把工作流指令当作程序源码处理,生成抽象语法树和控制流图。
- 运行时逐节点验证每步操作,成功率从50%提升至83.33%。
- 适合需要严格遵循复杂多步流程的自动化任务场景。
大型语言模型(LLM)代理越来越多地接收包含目标结果及允许步骤、分支和工具调用的自然语言工作流指令(如零售支付政策)。当这些指令作为提示上下文提供时,模型同时掌控流程选择与步骤执行。随着交互累积,代理可能跳过必要步骤、进入不支持的分支,或使用不支持的参数执行有效步骤——这种现象称为工作流错位。本文提出COVENANT,一种面向工作流对齐的代理执行编译器-解释器架构。核心思想是将工作流指令视为源程序而非提示。COVENANT将其转化为工作流抽象语法树(WAST),并降级为工作流控制流图(WCFG)。运行时,控制器逐节点解析WCFG,根据指令提取的要求验证每个提议,在提交控制器状态或推进图前进行检查,并返回诊断反馈以供修复。在三个现有基准的120个案例上评估,涵盖七种工作流场景。相比最先进LLM代理,COVENANT将基准成功率从50.00%提升至83.33%,工作流错位失败率从42.50%降至15.83%(相对降低62.75%)。结果表明,COVENANT显著缓解了工作流错位问题,推动大模型代理对齐从单一提示遵循迈向复杂多步流程的可靠执行。
原文摘要 · Abstract (English)
Large language model (LLM) agents are increasingly entrusted with natural-language workflow instructions (e.g., retail-payment policies) that specify not only what outcome to achieve, but also which steps, branches, and tool interactions are permitted. When these instructions are supplied as prompt context, however, the model retains control over both procedure selection and step execution. As interactions accumulate, an agent can skip required steps, take unsupported branches, or execute a valid step with unsupported arguments or effects--a failure mode we call workflow misalignment. In this work, we propose COVENANT, a compiler-and-interpreter architecture for workflow-aligned agent execution. Our key insight is to treat workflow instructions as source programs rather than prompts. COVENANT converts the instructions into a workflow abstract syntax tree (WAST) and lowers it to a workflow control-flow graph (WCFG). At runtime, a controller interprets the WCFG one node at a time, checks each proposal against requirements extracted from the instructions before committing controller state or advancing the graph, and returns diagnostic feedback for repair. To evaluate COVENANT, we use 120 cases from three existing benchmarks, spanning seven workflow scenarios. Compared with state-of-the-art LLM agents, COVENANT improves benchmark success from 50.00% to 83.33% and reduces the workflow-misalignment failure rate from 42.50% to 15.83% (62.75% relative). These results show that COVENANT substantially mitigates workflow misalignment, moving LLM-agent alignment beyond isolated prompt following toward reliable execution of complex and multi-step workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。