arXiv:2606.04536cs.AI2026-06被引 1

让大模型在单次任务中实时学习,通过轻量更新权重记住经验。

Scaling Self-Evolving Agents via Parametric Memory

论文配图:Scaling Self-Evolving Agents via Parametric Memory
图 1 · 摘自论文原文
  • 用可快速更新的参数化记忆替代纯文本记忆,实现行为动态调整。
  • 在多个评测集上超越传统方法,在不同规模模型下表现稳定提升。
  • 适合需要持续学习与快速适应的复杂任务场景,如长序列推理。

现有增强记忆的大语言模型代理将过往经验仅存储于提示词空间,以文本摘要或检索片段形式存在,且在推理过程中保持模型参数不变。这类代理只能‘查阅’已见内容,却无法‘从中学习’:其策略不受经验影响,任何超出上下文的信息将永久丢失。我们提出 exttt{TMEM},一种自演化参数化记忆框架,使代理不仅将历史压缩为显式记忆,还能通过轻量级在线更新,将提炼后的监督信号融入快速 LoRA 权重 $Δ_t$,真正改变单个任务中的后续行为。我们将此建模为具有快速权重滚动动态的智能体决策过程:动作从 $π_{θ_0+Δ_t}$ 中采样,而提取动作生成用于更新 $Δ_t$ 的监督信号。该视角使提取策略可直接通过强化学习优化:训练 $θ_0$ 不仅提升任务动作,也提高用于在线 LoRA 适配的数据质量。我们进一步提出基于 SVD 的 LoRA 子空间初始化方法,加速在线收敛。在 LoCoMo、LongMemEval-S、多目标搜索及 CL-Bench 上的实验表明, exttt{TMEM} 在不同模型规模下均持续优于基于摘要和基于检索的基线方法。

原文摘要 · Abstract (English)

Existing memory-augmented LLM agents store past experience exclusively in prompt space, as textual summaries or retrieved passages, while keeping model parameters frozen throughout a rollout. Such agents can \emph{look up} what they have seen but cannot \emph{learn from} it: their policy is unchanged by experience, and any information dropped from the context is permanently lost. We introduce \texttt{TMEM}, a self-evolving parametric memory framework in which the agent not only compresses history into explicit memory but also absorbs distilled supervision into fast LoRA weights $Δ_t$ via lightweight online updates, genuinely altering its future behavior within a single episode. We formalize this as an agentic decision process with fast-weight rollout dynamics: actions are sampled from $π_{θ_0+Δ_t}$, while extraction actions produce supervision that updates $Δ_t$ for subsequent decisions. This view makes the extraction policy directly optimizable by RL: training $θ_0$ improves not only task actions but also the quality of the data used for online LoRA adaptation. We further propose SVD-based initialization of the LoRA subspace to accelerate online convergence. Experiments on LoCoMo, LongMemEval-S, multi-objective search, and CL-Bench show that \texttt{TMEM} consistently outperforms summary-based and retrieval-based baselines across different model scales.

参数记忆在线学习LoRA智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。