通过三阶段协作避免自我确认陷阱,提升大模型代理的经验学习可靠性。
Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning

- 多智能体并行执行+第三方分析+共识验证,打破单代理闭环。
- 在三个长周期任务基准上超越强基线,成功率显著提升。
- 适合需要可靠长期自进化能力的复杂任务代理系统。
以经验驱动的自我演化对大语言模型代理在开放世界中的持续改进至关重要。然而,现有方法多依赖单代理循环:同一代理执行任务、总结结果并决定记忆内容,易陷入自我确认陷阱——错误但自洽的路径被误判为有效经验,导致检索与复用时累积误差。为此,我们提出执行-提炼-验证(EDV)框架,实现可靠的经验学习。在执行阶段,多个异构代理并行探索相同任务空间,生成多样化候选轨迹;在提炼阶段,第三方代理对比分析这些轨迹,生成候选经验,降低执行者中心的总结偏差;在验证阶段,执行组通过共识机制验证候选经验,仅通过验证的经验才存入共享或私有记忆。通过解耦三阶段,EDV将经验学习从孤立自我反思转变为协作构建,在写入记忆前过滤错误和噪声内容。我们在tau2-bench、Mind2Web和MMTB三个高难度长周期基准上评估,结果表明EDV持续优于强基线,验证了可靠经验构建对鲁棒代理自演化的关键作用。代码已开源:https://github.com/shidingz/EDV。
原文摘要 · Abstract (English)
Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction. However, existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self-Confirmation Trap: wrong-but-self-consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute-Distill-Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third-party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor-centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self-reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long-horizon benchmarks: tau2-bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self-evolution. Our code is available at https://github.com/shidingz/EDV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。