arXiv:2603.18272cs.AIcs.CL2026-03中稿 · EMNLP被引 5

让大模型通过检索经验来学习新任务,提升泛化能力。

Retrieval-Augmented LLM Agents: Learning to Learn from Experience

  • 将经验检索融入微调过程,实现上下文学习。
  • 在未见过的任务上表现优于现有方法,即使经验不完美。
  • 适合构建能从失败中学习的智能体,无需测试时更新参数。

尽管大语言模型推动了通用智能体的发展,但在未见任务上的鲁棒泛化仍具挑战。现有方法包括监督微调和无需训练的记忆增强生成,但各有局限:微调难以外推至新任务,而经验检索性能常低于监督基线。本文结合两种方法,研究检索增强型大模型智能体如何在上下文中利用检索到的经验轨迹进行学习。首先建立一个强基线LoRA微调方法,表现超越多个前沿智能体训练流程。其次分析经验检索的关键设计选择,包括存储、查询与轨迹筛选。随后将经验检索直接集成至微调流程,显著提升对未见任务的泛化能力。最后发现,这种增益在经验不完整的情况下依然有效,甚至在复用自身失败尝试而无测试时参数更新时也成立。结果表明,简单的情景式检索是智能体记忆的良好基础,而检索感知微调是构建可从经验中学习的智能体的实用高效框架。

原文摘要 · Abstract (English)

While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are supervised fine-tuning and training-free memory-augmented generation using retrieved experience; yet both have limitations: fine-tuning often fails to extrapolate to new tasks, while experience retrieval often underperforms compared to supervised baselines. In this work, we combine these approaches and study how retrieval-augmented LLM agents can learn to use retrieved trajectories in-context. First, we establish a strong LoRA fine-tuning baseline that outperforms several state-of-the-art agent training pipelines. Second, we analyze key design choices for experience retrieval, including storage, querying, and trajectory selection. We then integrate experience retrieval directly into the fine-tuning process, finding that this substantially improves generalization to unseen tasks. Finally, we show that these gains often persist with imperfect experience and, even when agents reuse their own failed attempts without test-time parameter updates. Overall, our results establish simple episodic retrieval as a strong foundation for agent memory and retrieval-aware fine-tuning as a practical and effective framework for building agents that learn to learn from experience.

智能体检索增强微调学习能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。