让大模型学会从过往搜索经验中反思，避免重复犯错。
R^2-Mem: Reflective Experience for Memory Search
- 用评分器和自反思学习器提炼历史搜索中的好坏经验。
- 实验显示F1提升22.6%，令牌消耗降低12.9%，搜索迭代减少20.2%。
- 适合需要自我改进的大型语言模型搜索系统使用。
深度搜索作为一种新兴范式，使智能体能在无需预先管理记忆的情况下检索细粒度历史信息。然而，现有深度搜索代理因未能从以往高质量与低质量搜索轨迹中学习，导致重复错误行为。为解决此问题，我们提出R^2-Mem，一种用于记忆搜索系统的反思经验框架。离线阶段，基于评分标准的评估器对历史轨迹中的高/低质量步骤进行打分，自反思学习器从中提炼出抽象经验；在线推理时，检索到的经验将引导后续搜索动作，避免重复错误并维持高质量行为。大量实验证明，R^2-Mem在强基线基础上持续提升有效性与效率，F1分数最高提升22.6%，令牌消耗减少12.9%，搜索迭代次数降低20.2%。结果验证了R^2-Mem是一种无强化学习、低成本的自提升大模型代理解决方案。
原文摘要 · Abstract (English)
Deep search has recently emerged as a promising paradigm for enabling agents to retrieve fine-grained historical information without heavy memory pre-managed. However, existing deep search agents for memory system repeat past error behaviors because they fail to learn from the prior high- and low-quality search trajectories. To address this limitation, we propose R^2-Mem, a reflective experience framework for memory search systems. In the offline stage, a Rubric-guided Evaluator scores low- and high-quality steps in historical trajectories, and a self-Reflection Learner distills the corresponding abstract experience. During the online inference, the retrieved experience will guide future search actions to avoid repeated mistakes and maintain high-quality behaviors. Extensive experiments demonstrate that R^2-Mem consistently improves both effectiveness and efficiency over strong baselines, improving F1 scores by up to 22.6%, while reducing token consumption by 12.9% and search iterations by 20.2%. These results verify that R^2-Mem provides a RL-free and low-cost solution for self-improving LLM agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。