arXiv:2609.02889cs.CL2026-09

优化大模型代理的提示词,发现控制机制是价值核心,平均分配资源反而适得其反。

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

论文配图:Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents
图 1 · 摘自论文原文
  • 将提示词拆分为角色、策略、工具规则和反思控制四部分分别优化
  • 只有反思控制模块带来显著提升,单独使用可使成功率提高11.9%
  • 均匀分配优化预算会冻结所有模块,集中资源才有效

越来越多研究通过演化提示词(harness)来提升冻结的大语言模型作为智能体的性能,现有方法通常将提示词视为单一字符串进行优化。本文提出HARNESSEVO,将提示词分解为角色、任务策略、工具/格式规则、反思/控制四个独立可演化的模块,并在相同预算下采用留一法分析各模块贡献。在使用7B模型的ALFWorld任务中,整体成功率未超过原始提示词(0.657)或扁平演化(0.642)。但槽级分析显示,几乎全部优化价值集中在反思/控制模块,其留一内增益达+0.119,其余模块无显著贡献。进一步发现,均匀分配64次回放至四个模块(每模块仅16次)低于优化器有效搜索阈值,导致各模块均冻结于初始状态;而将预算集中于高收益控制模块,仅用一半预算即实现0.761的成功率。该效应具任务依赖性:在WebShop任务中,所有模块均无变化,表明不存在可重复的可言说型控制失败,而非预算不足所致。结果表明,提示词价值高度局部化,均匀预算分配可能有害,结构化演化前应先完成信用分配。

原文摘要 · Abstract (English)

A growing body of work improves frozen large language models (LLMs) as agents by evolving their harness: the textual scaffolding around the model, including persona, strategy, format rules, and control heuristics. Existing reflective prompt-evolution methods usually optimize this harness as one flat string. We instead ask where the optimization value actually resides. We introduce HARNESSEVO, which decomposes the harness into four separately evolvable slots: role, task-strategy, tool/format-rules, and reflection/control. Using the same reflective optimizer under an iso-budget setting, we pair this decomposition with leave-one-in and leave-one-out attribution to measure the contribution of each slot. On ALFWorld with a frozen 7B backbone, HARNESSEVO does not significantly improve the overall binary success rate over either the stock harness or flat-string evolution: 0.657 versus 0.642 and 0.642, respectively. However, the slot-level analysis reveals that nearly all useful optimization value is localized in the reflection/control slot, which achieves a leave-one-in gain of +0.119. The other slots are individually null. We further show that uniform budget splitting is harmful: allocating 64 rollouts across four slots leaves only 16 per slot, below the optimizer's effective search floor, causing every slot to freeze at its empty seed. Concentrating the budget on the high-credit control slot recovers the lost gain, reaching 0.761 with half the split budget. The effect is task-contingent. On WebShop, all slots freeze empty and all methods tie, indicating a genuine absence of recurrent, verbalizable control failures rather than budget starvation. Overall, our results suggest that harness value is localized, uniform budget splitting can be actively harmful, and credit assignment should precede structured agent-evolution.

提示词演化代理优化控制机制预算分配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。