用自然语言反馈优化文本策略，发现生成有效策略比执行更难。
From Agent Failures to Text Policies: What Works and What Breaks
- 用人类反馈作为文本优化的梯度，不修改模型权重。
- 人类编写的策略使7B模型在TextWorldExpress上提升5.0分成功率。
- 从智能体轨迹生成策略效果不佳，关键瓶颈是策略生成与选择。
TextGrad通过反馈修正文本，其核心思想是自然语言反馈可作为无权值更新的文本优化梯度。应用于智能体时，反馈仅在动作序列结束后到达，难以定位失败原因。本文区分了遵循有效策略的能力与从经验中学习策略的能力。主要发现是二者存在明显差距：人类编写的策略使两个冻结的7B智能体在TextWorldExpress上成功度提升5.0分，表明有效策略文本存在。然而，即使使用更丰富的轨迹、反事实证据或迭代GEPA搜索，由智能体轨迹生成的策略也无法稳定超越固定提示。因此，智能体级TextGrad的核心挑战并非执行文本更新，而是从经验中可靠生成并选择策略。
原文摘要 · Abstract (English)
TextGrad improves language-model systems by revising text from feedback. Its core thesis is that natural-language feedback can act as a gradient for optimizing text components without changing model weights. Applying it to agents is harder because feedback arrives only after a sequence of actions, making it difficult to identify which decision caused failure. We study this problem by separating the ability to follow a useful policy from the ability to learn that policy from experience. Our main finding is a clear gap between these two abilities. Human-written policies improve two frozen 7B agents on TextWorldExpress by 5.0 success points, showing that useful policy text exists. However, policies generated from agent trajectories do not reliably outperform fixed prompting, even with richer traces, counterfactual evidence, or iterative GEPA search. The main challenge for agent-level TextGrad is therefore not executing textual policy updates, but reliably generating and selecting them from experience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。