让大模型像人一样持续学习,通过记忆增强实现更智能的知识积累。
From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

- 基于个人化页面排名算法,融合深度段落整合与在线大模型使用。
- 在事实、理解与关联记忆任务上全面超越标准RAG,关联记忆提升7%。
- 适合需要持续更新知识的大模型应用,如智能助手与知识库系统。
人类持续获取、组织和利用知识的能力是智能的关键特征,人工智能系统需逼近此能力以释放全部潜力。针对大语言模型(LLMs)持续学习的挑战,检索增强生成(RAG)已成为引入新信息的主要方式。然而,其依赖向量检索难以模拟人类长期记忆的动态与互联特性。近期的RAG方法通过引入知识图谱等结构来弥补感知与关联性缺陷,但在基础事实记忆任务上的表现显著下降。本文提出HippoRAG 2,解决这一意外性能下滑问题,在事实、意义理解与关联记忆任务上均优于标准RAG。该框架基于原始HippoRAG中的个性化页面排名算法,增强段落深度整合与大模型的高效在线使用。该组合使RAG系统更接近人类长期记忆效果,在关联记忆任务上比当前最优嵌入模型提升7%,同时在事实知识与意义理解能力上也表现更优。本工作为大模型的非参数持续学习开辟了新路径。代码与数据见:https://github.com/OSU-NLP-Group/HippoRAG。
原文摘要 · Abstract (English)
Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。