让智能体像人一样记经验，不改模型也能持续学习。
APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay
- 用结构化知识图谱存任务全过程，按语义、操作模式和图路径检索。
- 在多个测试中比无记忆基线提升7.6个百分点，冻结模型仍有效。
- 适合需要长期积累经验的自主智能体，如编程或复杂交互场景。
LLM智能体在每个任务中都重复完整推理，即使刚解决过类似问题。我们提出非参数化经验记忆 APEX-EM，将完整的程序-情景轨迹存储于类型化的程序知识图（PKG）中，并通过三种方式检索：语义搜索、抽象操作序列的结构签名匹配，以及图遍历。采用计划-检索-生成-迭代-吸收（PRGII）流程，对经验进行质量控制并提交，同时索引成功与失败案例，使智能体学会复用什么、避免什么。部署期间不更新模型权重。我们在五个基准上评估：BigCodeBench、KGQAGen-10k、HLE、Lifelong Agent Bench 和 ALFWorld。由于先前工作使用不同主干，我们采用相同主干比较，固定模型能力。在共享 GPT-4o 主干的 BigCodeBench 外部迁移测试中，APEX-EM 相比无记忆基线提升 +7.6 个百分点，是 MemRL 的 +2.3 个百分点的 3.3 倍。在共享 GPT-4o-mini 主干的 Lifelong Agent Bench 上，累积成功率提升 +1.4 个百分点（OS）和 +1.0 个百分点（DB）。在 KGQAGen-10k 上，冻结记忆在 1,079 道盲测题上达到 73.7%，而无记忆仅为 42.0%，接近手握真值子图的基准（84.9%）。在三个 Opus 规模下，记忆增益稳定在 +27 至 +32 个百分点，说明其增强而非替代模型能力。组件分析表明：单一机制不主导；教师反馈对代码影响小但对结构化查询加成 +10.3 个百分点；结构签名检索的迁移效果是仅语义检索的 3.3 倍；当丰富反馈不可用时，单轮内迭代可恢复大部分收益。结果支持按领域模块化构建记忆。
原文摘要 · Abstract (English)
LLM agents rerun full reasoning for every task, even one they solved moments earlier. We introduce \textbf{APEX-EM}, a non-parametric experience memory that stores complete procedural-episodic traces in a typed Procedural Knowledge Graph (PKG) and retrieves them through three channels: semantic search, structural-signature matching over abstract operation sequences, and graph traversal. A Plan-Retrieve-Generate-Iterate-Ingest (PRGII) workflow produces, quality-gates, and commits experiences, indexing both successes and failures so the agent learns what to reuse and what to avoid. No weights change during deployment. We evaluate on five benchmarks: BigCodeBench, KGQAGen-10k, HLE, Lifelong Agent Bench, and ALFWorld. Because prior work uses different backbones, we base our claims on same-backbone comparisons that hold model capability fixed. On held-out BigCodeBench transfer with a shared GPT-4o backbone, APEX-EM gains +7.6\,pp over the no-memory baseline, $3.3\times$ MemRL's +2.3\,pp under the identical setup. On Lifelong Agent Bench with a shared GPT-4o-mini backbone, it gains +1.4\,pp (OS) and +1.0\,pp (DB) cumulative success. On KGQAGen-10k, frozen memory transfers to a blind 1{,}079-question test split at 73.7\% versus 42.0\% with no memory, approaching an oracle handed the ground-truth subgraph (84.9\%). Across three Opus scales the memory gain stays at +27 to +32\,pp, so it adds to model capability rather than substituting for it. Component analysis shows no single mechanism dominates: teacher feedback is negligible for code but adds +10.3\,pp on structured queries, structural signatures give $3.3\times$ the transfer of semantic-only retrieval, and within-epoch iteration recovers most of the gain when rich feedback is unavailable. These results argue for modular memory composed per domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。