arXiv:2609.05736cs.AI2026-09

不重训练模型,通过优化工具调用框架提升大模型代理性能

Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses

论文配图:Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses
图 1 · 摘自论文原文
  • 通过修改提示词和工具边界中间件来优化代理运行时架构
  • 在三个数据集上平均提升10.1至14.9个百分点,且可靠性指标优异
  • 适合关注代理鲁棒性与可复现性的研发人员使用

大模型工具代理可通过修改运行时框架(如提示词、工具接口、中间件、状态处理和恢复逻辑)进行改进,无需重新训练模型。本文将此问题建模为固定模型多轮代理的资源受限框架选择,搜索空间限定于提示词和工具边界中间件。优化操作为工具边界处的受控拦截,而非任意改写执行逻辑。提出一种无特定优化器依赖的协议,报告平均保留提升、最差条件提升、可重复性、成本诊断及RelLift95(B),后者是在预算B下对保留增益的保守估计。在BFCL多轮、tau2-Retail和tau2-Telecom数据集上,包含PRISM在内的优化器实现10.1至14.9个百分点的平均保留提升,并在所有任务中获得正向经验性RelLift95。组件消融显示主要收益来自失败面路由和编辑模式约束。

原文摘要 · Abstract (English)

LLM tool agents can be improved without retraining by modifying the runtime harness around a fixed model: prompts, tool interfaces, middleware, state handling, and recovery logic. We study this setting as resource-bounded harness selection for fixed-model multi-turn tool agents, with the search surface scoped to prompts and tool-boundary middleware: edits are guarded intercepts at the tool boundary, not arbitrary rewriting of agent execution logic. Our optimizer-agnostic protocol reports mean held-out lift, worst-condition lift, repeatability, logged cost diagnostics, and RelLift95(B), a conservative estimate of the held-out gain of the harness selected under budget B. We instantiate the protocol with prompt-only and prompt-plus-middleware optimizers, including PRISM, which clusters failures and routes repairs to prompt, tool-boundary middleware, or joint edit surfaces within a Pareto search. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM obtains mean held-out lifts of 14.2, 14.9, and 10.1 percentage points and positive empirical RelLift95 on all three benchmarks, and a component ablation attributes the margin chiefly to failure-surface routing and the edit-pattern constraint. Across optimizers, the results show that some search procedures can occasionally find large gains but still choose brittle updates, so the reliability of the chosen harness should be reported alongside average held-out lift.

大模型代理工具调用性能优化鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。