轻量记忆管理未必比直接检索原始对话更优,效果取决于检索器和上下文预算。
Reproducing LightMem: Naive RAG Is Just as Good for Memory Management

- 用原始对话轮次直接检索,对比构造记忆的LightMem方法
- 更换检索器使准确率在58.1%至75.5%间波动,差距显著
- 紧约束下构造记忆有优势,但整体不优于原始检索
长期对话智能体需要访问早期交互信息,如用户偏好或提及事实。重复提供完整对话历史成本高昂,因此许多方法将过往交互转化为紧凑条目以供检索。LightMem是一种近期轻量级记忆管理方法,报告了高效且低成本的表现。然而,它仍依赖独立构建的记忆表示,并仅使用单一检索器评估,未明确其对检索器选择的敏感性,也未验证记忆构建是否丢弃关键答案信息。本研究复现LightMem,与直接从原始用户轮次检索的Naive RAG进行比较。我们重现了LightMem的主要配置趋势,但发现检索器选择是性能变化的关键:固定记忆存储时更换检索器,准确率从58.1%升至75.5%。构造记忆并未持续优于原始轮次检索。在匹配检索深度下,Naive RAG表现更好;而仅在严格回答令牌预算下,LightMem占优。通过理想评估进一步显示,记忆构建会移除部分答案相关的信息。总体而言,LightMem仅提供上下文效率的权衡,而非对Naive RAG的普遍优势,其价值取决于检索器和可用令牌预算,推动未来在检索、重排序、查询设计及其与原始/构造记忆表示交互方面的研究。
原文摘要 · Abstract (English)
Long-term conversational agents require access to information from earlier interactions, such as a user's preferences, past requests, or previously mentioned facts. Repeatedly providing the full dialogue history can be expensive as conversations grow, so many memory approaches instead transform past interactions into compact entries that can be retrieved when needed. LightMem is a recent lightweight memory-management approach that reports strong effectiveness while maintaining relatively low construction cost. However, it still relies on a separate constructed memory representation and is evaluated with only one retriever, leaving unclear how sensitive its results are to retriever choice and whether memory construction discards answer-relevant information. In this study, we reproduce LightMem and compare it with Naive RAG, which retrieves directly from raw user turns. We recover LightMem's main configuration trend, but find that retriever choice is a major source of performance variation: changing only the retriever over a fixed LightMem store shifts answer accuracy from 58.1% to 75.5%. Constructed memories also do not consistently outperform raw-turn retrieval. Naive RAG generally performs better at matched retrieval depths, whereas LightMem performs better mainly under tight answering-token budgets. Oracle evaluation further shows that memory construction removes some answer-relevant information. Overall, LightMem offers a context-efficiency trade-off rather than a general advantage over Naive RAG. Its value depends on the retriever and available token budget, motivating future work on retrieval, reranking, query formulation, and their interaction with raw and constructed memory representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。