arXiv:2606.01528cs.AI2026-06

让智能体通过新奇信号同步学习记忆与探索,提升长期自主能力。

Joint Agent Memory and Exploration Learning via Novelty Signals

论文配图:Joint Agent Memory and Exploration Learning via Novelty Signals
图 1 · 摘自论文原文
  • 用新奇信号驱动记忆与探索联合训练,形成自增强循环。
  • 在未见过环境中表现优异,探索深度接近闭源模型,但消耗更少令牌。
  • 无需人工标注,利用代码覆盖率等确定性信号实现无监督监督。

在开放环境中的自主智能体需要有效探索,但当前语言模型智能体在此方面表现不佳。有效探索依赖记忆,然而保留原始交互历史会带来巨大计算开销。虽然潜在记忆可压缩历史信息,但其训练缺乏可靠的监督信号。本文提出联合智能体记忆与探索学习(JAMEL)框架,通过新奇驱动的交互联合训练记忆模块与探索策略。我们发现记忆与探索存在相互依赖关系:持续探索需记忆区分已耗尽行为与新行为,而新奇探索则为记忆提供未来探索所需的监督信号。通过利用图形用户界面领域中确定且持久的新奇信号(如代码覆盖率),我们实现了自然、无需标注的监督。实验表明,该方法能成功泛化至未见过的环境,其探索能力超越开源基线,并接近闭源模型的探索深度,同时显著降低令牌消耗。代码与模型已在 https://github.com/MobileLLM/JAMEL 开源。

原文摘要 · Abstract (English)

In open-ended environments, exploration is fundamental for autonomous agents, yet current language model agents struggle with this. Effective exploration requires memory, but retaining raw interaction histories is computationally expensive over long trajectories. While latent memory offers a solution to compress interaction histories, its training lacks reliable supervisory signals. We introduce \textbf{J}oint \textbf{A}gent \textbf{M}emory and \textbf{E}xploration \textbf{L}earning (\textbf{JAMEL}), a framework that trains agentic memory and exploration policy together through novelty-driven interaction. We observe that memory and exploration form a mutually dependent loop: sustained exploration requires memory to distinguish exhausted behaviors from unseen ones, while novelty-seeking interaction provides the supervision needed to make memory useful for future exploration. By utilizing deterministic and persistent novelty signals such as code coverage in the GUI domain, we provide natural, annotation-free supervision for the memory module. Empirical evaluations demonstrate that \ours successfully generalizes to unseen environments. Its exploration capability outperforms open-weight baselines and rivals the exploration depth of a closed-source model while reducing token consumption. Our code and model are open-sourced at https://github.com/MobileLLM/JAMEL.

智能体记忆机制探索学习新奇信号

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。