arXiv:2608.04003cs.CL2026-08被引 2

测试个人智能体能否通过积累经验持续自我改进。

PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

论文配图:PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents
图 1 · 摘自论文原文
  • 设计新基准PAST-Bench,对比有无记忆时的性能差异。
  • 7个模型中改进效果不一,仅部分任务体现预期优化路径。
  • 提出Hermes+框架,提升记忆利用效率,尤其擅长更新过时信息。

递归自我改进要求智能体将积累的经验转化为未来更优行为。个人AI代理因其在会话间保留偏好、任务历史、工具流程和学习技能,成为研究该能力的理想场景。然而,现有方法尚未系统检验经验留存是否真能带来持续提升。本文提出PAST-Bench,一个专门用于隔离此问题的基准测试。每个代理在相同条件下运行26种场景、204个任务,分组开启或关闭记忆功能。结果既报告后续任务的性能增益,也检验该增益是否符合预设的保存-检索-更新路径。在七个基础模型和四种代理框架中,改进确实存在但分布不均。同为显著提升的模型,其改善机制差异明显。基于此发现,我们开发了Hermes+,在代理循环的五个阶段引入针对性干预。Hermes+显著提升平均收益,并增强路径证据,尤其在需替换过时状态的任务中表现突出,但效果仍依赖具体能力与模型。PAST-Bench与Hermes+共同构建了评估与诊断持久性代理如何从经验留存走向系统性改进的基线。

原文摘要 · Abstract (English)

Recursive self-improvement requires agents to turn accumulated experience into better future behavior. Personal AI agents offer a concrete setting for studying this capability because they retain preferences, task histories, tool routines, and learned skills across sessions. Yet whether retained experience actually improves them over time has not been systematically tested. We introduce PAST-Bench, a benchmark designed to isolate this question. Each agent runs through ordered sequences of fresh-session tasks under matched conditions that turn retained experience on and off. It spans 26 scenarios and 204 episodes across memory, procedural reuse, information gathering, and update. We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway. Across seven base models and four agent frameworks, improvement is real but uneven across capabilities. Agents with the same headline gain can differ markedly in whether that gain is supported by evidence of the intended pathway. Guided by these findings, we develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop. Hermes+ raises the average gain from retained experience and provides clearer pathway evidence, with its strongest improvement on tasks requiring outdated state to be replaced, although the effect remains capability- and model-dependent. Together, PAST-Bench and Hermes+ provide an evaluation and diagnostic foundation for studying how persistent agents can progress from retaining experience to systematically improving through it. Code: https://github.com/Gen-Verse/PAST-Bench

智能体自我改进基准测试记忆机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。