让大模型代理更稳定，减少执行失败的波动性。
Closing the Consistency Gap: Self-Evolving Agents That Learn to Stay on Course
- 通过分析轨迹不一致点，自动提取经验并存入记忆。
- 在相同任务上五次全成功率提升至69%，跨任务也提升至66%。
- 适合追求高可靠性的实际部署场景，如自动化流程系统。
基于大语言模型的智能体虽然平均表现准确，但在实际应用中却存在可靠性问题。例如，在AppWorld基准上使用GPT-4.1的ReAct代理，尽管单次成功率平均达77%，但重复五次任务全部成功的比例仅为53%，两者之间存在24个百分点的差异，称为一致性差距。本文提出一种自演化代理框架，通过一致性分析器识别轨迹中不稳定、低一致性的步骤，并将其转化为可复用的指导规则存入记忆。该框架在相同任务上的五次全成功率提升16个百分点，在相似任务泛化上提升13个百分点，显著缩小了性能波动，为可信智能体部署提供了关键支撑。
原文摘要 · Abstract (English)
Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains largely unaddressed. When given the same task five times, a ReAct agent on the AppWorld benchmark using GPT-4.1 succeeds in all five runs only 53% of the time, even though its per-run pass rate averages 77%. We call this 24-point shortfall the consistency gap, and we argue that addressing it is a precondition for trustworthy AI agent deployment. We present a self-evolving agent framework that reduces this gap by identifying unstable, low-consistency steps in agent trajectories and converting them into episodic memory the agent can draw on in future runs. At its core is a Consistency Analyzer that pinpoints where and why a trajectory is likely to flip across executions, and a Guideline Generator that converts the diagnosis into targeted guidelines, committed to memory and injected into future agent executions on similar tasks. On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。