arXiv:2511.14584cs.LGcs.AI2025-11中稿 · ICML

让大模型智能体在任务中出错后自动纠错,无需示例。

ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing

  • 用快慢双流程切换机制,错后自动诊断并修正。
  • 在ALFWorld上使Qwen-3-8B成功率从35.1%升至75.4%。
  • 适合需要高鲁棒性的自主智能体开发场景。

我们提出ReflexGrad,一种无需示例的、用于大模型智能体单轮任务中失败恢复的双流程架构。当智能体早期选错路径且耗尽步数时,事后轨迹仍蕴含逃生信息——但现有架构无法在单次任务内利用。ReflexGrad在每3步进行一次快速文本优化(TextGrad风格),当连续5次进度得分偏低时触发慢速因果诊断(Reflexion风格),激活后生成可复现的触发信号、诊断结果和验证修复。该机制通过确定性优先合并保持自然语言策略连贯性。在无演示条件下,10次随机种子测试中,Qwen-3-8B在ALFWorld 134个任务上的表现从35.1%提升至75.4%(+40.3个百分点),超越计算量相当的1次提示LATS(+2.7pp, p≈0.01)、ToT(+5.7pp, p<10⁻⁴)和Self-Refine(+6.7pp, p<10⁻⁵);GPT-5则从46.3%升至88.1%(+41.8pp)。跨模型差异仅1.5pp,接近种子噪声水平(p≈0.13),表明性能提升主要源于路由机制而非模型规模。代码、提示、每种子日志及敏感性分析均已公开。

原文摘要 · Abstract (English)

We present ReflexGrad, a dual-process architecture for within-episode failure recovery in LLM agents without demonstrations. When agents commit to a wrong approach early and exhaust the step budget, the post-failure trajectory contains the information to escape -- but no published architecture acts on it within a single episode. ReflexGrad routes between a fast process (TextGrad-style continuous refinement every $k{=}3$ steps) and a slow process (Reflexion-style causal diagnosis when $m{=}5$ consecutive low-progress scores fire a routing gate). A deterministic priority merge keeps the natural-language policy coherent, and each slow activation emits three observable artifacts: a reproducible trigger, a causal diagnostic, and a verified fix. On ALFWorld 134 tasks, $n{=}10$ seeds, no demonstrations, ReflexGrad lifts Qwen-3-8B from $35.1\%$ to $75.4\%$ ($+40.3$pp), beating compute-matched 1-shot LATS by $+2.7$pp ($p{\approx}0.01$), ToT by $+5.7$pp ($p{<}10^{-4}$), and Self-Refine by $+6.7$pp ($p{<}10^{-5}$); on GPT-5 the lift is $46.3{\to}88.1\%$ ($+41.8$pp). The $1.5$pp cross-model difference is within seed noise ($p{\approx}0.13$), suggesting that the routing mechanism, rather than model scale, is the primary source of the gain. Code, prompts, per-seed logs, and sensitivity sweeps are released.

智能体错误恢复大模型强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。