将技能文档转为带类型签名的伪代码,提升大模型代理执行效率
Skill-as-Pseudocode: Refactoring Skill Libraries to Pseudocode for LLM Agents
- 自动将自由文本技能库转换为带类型约束的伪代码
- 在ALFWorld上胜率从47%提升至82%,节省23%输入词和15%调用次数
- 适合需要高可靠性任务执行的智能体系统开发者
LLM代理使用的Markdown技能库以自由文本形式存在,导致代理每次调用都需重新推导输入格式和具体调用语法,陷入‘困惑→重查→仍困惑’的循环。本文提出Skill-as-Pseudocode(SaP),将Markdown技能库自动转化为带类型签名的伪代码,并通过四重确定性验证器(覆盖、绑定、替换、风险)筛选合格契约。经重构后的技能骨架同时包含类型化接口与可复用的动作模板,使代理获得‘做什么’与‘如何做’双重信号。在ALFWorld未见测试集(134场游戏,gpt-4o-mini,三组随机种子)中,SaP胜率为82/402,优于基线图技能(GoS)的47/402(合并麦克内马尔检验p=8.2×10⁻⁵),每局减少22.8±6.4%输入词和14.5±4.1%模型调用次数。消融实验表明,类型契约与具体模板的结合是关键增益来源,单独使用契约反而不如原始文本。
原文摘要 · Abstract (English)
Markdown skill libraries for LLM agents ship as free-form prose, forcing the agent to re-derive both the input schema and the concrete invocation syntax on every retrieval. This produces a "confused $\to$ re-retrieve $\to$ still confused" loop: the agent issues a partially-correct action, receives uninformative feedback, and re-retrieves the same prose. We propose Skill-as-Pseudocode (SaP), an automatic conversion of markdown skill libraries into typed pseudocode with deterministic quality control. From each cluster of similar procedural passages, SaP extracts a typed contract and filters it through a four-check deterministic verifier (coverage, binding, replacement, risk). Promoted contracts are inlined into a rewritten skill skeleton alongside restored action templates, giving the agent two complementary signals: a typed signature for what a skill does and a concrete template for how to invoke it. On the ALFWorld unseen split (134 games, gpt-4o-mini, three seeds), SaP wins 82/402 paired games versus 47/402 for the Graph-of-Skills (GoS) baseline (pooled McNemar $p = 8.2 \times 10^{-5}$), at $-22.8 \pm 6.4$% input tokens and $-14.5 \pm 4.1$% LLM calls per game. A bundle-component ablation attributes the gain to the pairing of typed contracts with concrete action templates: the contract alone falls below the prose baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。