通过隔离验证机制实现安全自进化，让AI agent持续优化但不退步。
Recursive Self-Evolving Agents via Held-Out Selection
- 用三层自然语言状态递归更新策略、技能和流程，仅保留不退步的改进版本。
- 在四个任务中表现最佳，ALFWorld单次通过率达69.3%，重试后达79.4%。
- 严格隔离验证防止性能下降，适合需要稳定迭代的复杂任务系统。
大模型代理通过演化自然语言产物（如反思、工作流、提示优化）而非更新权重来提升性能。本文提出递归自进化代理RSEA，其包含三层次自然语言状态：指令性策略、可复用技能与程序化手册。每代中，RSEA基于自身轨迹重写三部分，并仅在不劣于独立隔离测试集的情况下才保留更新，采用严格的‘更好才保留’机制。在四个不同基准（ALFWorld、GAIA、(τ)-bench、WebShop）上，对比六个基线方法（ReAct、Reflexion、GEPA、AWM、ACE、Dynamic Cheatsheet），均使用同一本地骨干模型。结果表明：第一，无通用最优方案；RSEA在ALFWorld单次通过率达69.3%（优于ReAct的64.6%，p=0.015），重试达79.4%，为最高分；但具体工作流生成（AWM）在强基底工具任务上更优。第二，无约束上下文演化高方差且危险；Dynamic Cheatsheet在ALFWorld达70.7%，但在WebShop降至0.14（ReAct为0.43）。第三，RSEA的隔离选择机制确保了递归演化的单调安全性：任何基准上均未显著劣于原始代理，当演化内容有害时自动回退至原版ReAct。
原文摘要 · Abstract (English)
LLM agents are increasingly improved without weight updates by evolving a natural-language artifact, such as reflections, workflows, playbooks, cheatsheets, or optimized prompts, that conditions a frozen policy. Such methods are typically reported as wins on the single benchmark where they help. We study them apples-to-apples and surface a sharper picture. We introduce RSEA, a Recursive Self-Evolving Agent that carries a compact three-layer natural-language state: an imperative strategy, reusable skills, and a procedural playbook. Across generations, RSEA rewrites all three layers from its own trajectories and commits a candidate only if it does not regress on a disjoint held-out split, using a strict keep-better gate. Across four diverse benchmarks, ALFWorld, GAIA, (τ)-bench, and WebShop, and six faithful baselines, ReAct, Reflexion, GEPA, AWM, ACE, and Dynamic Cheatsheet, all evaluated on one shared local backbone, we find three main results. First, no artifact universally wins. RSEA is the strongest single-pass method on ALFWorld, reaching 69.3% compared with 64.6% for ReAct (McNemar (p=0.015)), and reaches 79.4% with retry, the best overall result. However, concrete-workflow induction, represented by AWM, is best on the strong-backbone tool-use tasks. Second, unguarded context evolution is high-variance and unsafe. Dynamic Cheatsheet, which curates context online without a held-out gate, is near-best on ALFWorld at 70.7%, yet collapses on WebShop, with a score of 0.14 compared with 0.43 for ReAct. Third, RSEA's strict held-out selection is what makes recursive self-evolution monotone-safe: it never significantly underperforms the base agent on any benchmark and falls back to vanilla ReAct when evolved context would hurt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。