将大模型智能体的混乱操作记录提炼为高效确定性流程。
TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows

- 通过技能引导挖掘噪声轨迹,仅在有唯一证据时建立工具依赖关系。
- 在T1数据集上依赖识别达0.943召回率,减少API调用至原32%。
- 可识别不可逆副作用,拒绝错误编译,适合自动化流程重构场景。
使用工具的大语言模型智能体反复重蹈已执行过的步骤,生成混合了可复用结构、重试、探索、偶然顺序和重复查询的轨迹。我们提出TraceCompiler,一个技能引导的系统,从噪声轨迹中挖掘聚类并编译为可执行的、大部分确定性的工作流。仅当消费者参数包含可唯一归因于前驱生产者的值时才允许工具间依赖;每个硬边均携带可审计的证据元组,模糊关系被标记为可疑且不施加排序约束。绑定分为常量、用户输入、复制输出、转换或残余大模型决策。在T1上,机械化的规则在15,775个定义-使用边中实现0.928精确率与0.943召回率,优于邻接法(F1=0.711)和频率阈值直接跟随法(F1=0.712)。盲测编译技能在250条边达到0.992精度。在AppWorld中,通过确定性模拟器回放释放的轨迹以恢复掩码返回值,并对563个令牌边进行评估,精度达0.993,构成自洽性验证。成功编译两个常见意图:Venmo转账请求将34次观测调用压缩至11次运行调用,在留一法测试中通过15/21项,失败因所需分支未被观察到;Spotify/Todoist意图被正确拒绝编译,因其存在未明确定义的不可逆副作用。我们测量了调用减少效果,但未评估离线编译成本,因此不宣称整体效率提升。
原文摘要 · Abstract (English)
Tool-using language-model agents repeatedly rediscover procedures they have already executed, producing traces that mix reusable structure with retries, exploration, accidental ordering, and repeated lookups. We present TraceCompiler, a skill-guided system that mines clusters of noisy agent traces and compiles them into executable, mostly deterministic workflows. It admits an inter-tool dependency only when a consumer argument contains a value attributable uniquely to an earlier producer; every hard edge carries an auditable evidence tuple, and ambiguous relations are marked suspected and impose no ordering constraint. Bindings are classified as constants, user inputs, copied outputs, transforms, or residual LLM decisions. On T1, a mechanized form of the rule recovers producer-consumer dependencies at 0.928 precision and 0.943 recall over 15,775 def-use edges of its training split, against 0.711 F1 for adjacency and 0.712 for a frequency-thresholded directly-follows measure on identical data; the compiler skill run blind reaches 0.992 on 250 of those edges. On AppWorld we replay released trajectories in the deterministic simulator to recover masked return values and measure the rule against 563 token edges at 0.993 precision - a self-consistency check, since replay injects tokens by a related heuristic. We compile two recurring intents: a Venmo money-request intent reduces 34 observed API calls to 11 runtime calls and, under leave-one-out execution against the benchmark's own state tests, passes 15 of 21, the failing fold escalating rather than acting because its required branch was never observed; and a Spotify/Todoist intent the compiler correctly refuses to compile, because an irreversible side effect is under-determined. We measure call reduction but not offline compilation cost, so we claim no net efficiency result.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。