arXiv:2606.08049cs.AIcs.MA2026-06被引 2

让AI代理工作流自动判断何时用代码、何时用语言,提升长期可靠性。

SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows

论文配图:SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows
图 1 · 摘自论文原文
  • 根据执行证据动态决定步骤用代码还是自然语言,支持灵活调整。
  • 在WebArena上单轮成功率53.7%,三轮后仍保留91.7%成功任务,优于基线15.5个百分点。
  • 适合需要长期稳定运行的自动化任务,如网页操作与系统迁移。

AI代理正将过往经验转化为可复用的代码、工作流和程序记忆。复用虽能提升效率,但会带来生命周期可靠性问题:曾成功一次的产物可能因环境漂移、任务描述不全或分布变化而失效,尤其在网页自动化中。本文提出SKILL.nb框架,通过证据校准的生命周期策略管理可复用工作流。该框架采用选择性形式化:执行证据决定哪些步骤应转为可执行代码,哪些保持自然语言引导,以及何时需修正决策。工作流以可审计、版本化的笔记本形式存储,包含自然语言指导、多语言可执行单元、验证门控、回退路径及多模态证据(如输出、截图、错误轨迹)。运行时,门控条件执行机制使每个步骤仅在门控验证后运行代码,否则本地回退。在WebArena-Verified上,SKILL.nb实现53.7%单轮成功率,比最强基线高3.9个百分点;三次重执行后仍保持91.7%初始成功任务,领先第二名15.5个百分点。在有限修复下,可恢复72.9%后续失败,且修复后回归率仅4.2%,远低于基线的15.0%-17.0%。在Mind2Web跨网站与跨域测试中也表现最优。在GitLab迁移实验中,复用冻结于GitLab 15.7的状态,在目标版本GitLab 16.11和18.9上分别仅落后1.7分和超出0.6分,验证了其稳定性。结果表明,生命周期治理与门控执行是超越单次任务成功的可靠新维度。

原文摘要 · Abstract (English)

AI agents increasingly turn past experience into reusable artifacts such as code, workflows, and procedural memories. Reuse can improve efficiency, but it also creates a lifecycle reliability problem: artifacts that succeed once may fail under environment drift, underspecified tasks, or changing task distributions, especially in web automation. We introduce SKILL.nb, a framework for governing reusable agent workflows with evidence-calibrated lifecycle policies. SKILL.nb uses selective formalization: execution evidence decides which workflow steps should become executable code, which should remain natural-language guided, and when those choices should be revised. Workflows are stored as auditable, versioned notebooks that interleave natural-language guidance, multi-language executable cells, validation gates, fallback paths, and multimodal evidence such as outputs, screenshots, and error traces. At runtime, gate-conditioned execution lets each step run code when its gates validate, or fall back locally when drift invalidates the executable realization. On WebArena-Verified, SKILL.nb achieves 53.7% single-round success, improving over the strongest baseline by 3.9 percentage points. Across three re-executions, it retains 91.7% of initially successful tasks, 15.5 points above the next best method. Under bounded repair, it recovers 72.9% of subsequent failures while limiting post-repair regressions to 4.2%, compared with 15.0% to 17.0% for persistent baselines. It also leads on Mind2Web cross-website and cross-domain splits. In a GitLab migration test, SKILL.nb preserves performance when reusing frozen state learned on GitLab 15.7, with frozen-versus-fresh target-version gaps of -1.7 points on GitLab 16.11 and +0.6 points on GitLab 18.9. These results identify lifecycle governance and gate-conditioned execution as reliability axes beyond one-shot task success.

AI代理工作流可靠性自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。