提出在线更新RAG模型方法，实现知识动态融合与实时适应。
Research on the Online Update Method for Retrieval-Augmented Generation (RAG) Model with Incremental Learning
- 用动态记忆捕获新数据，通过可调知识蒸馏逐步融合进核心模型。
- 引入分层索引与多层门控机制，提升检索内容的准确性和针对性。
- 多阶段网络结构支持新旧知识迭代更新，适合需持续学习的应用场景。
在信息技术飞速发展和数据量激增的背景下，语言模型面临难以及时适应新知识的挑战。本文提出一种基于现有RAG模型的在线更新方法，包含多项创新机制：首先利用动态记忆捕获新数据样本，并通过可调知识蒸馏策略逐步融入核心模型；同时，在检索模块中引入分层索引与多层门控机制，提升检索结果的精准性；最后在生成阶段构建多阶段网络结构，对各阶段中间表示进行交叉注意力匹配与筛选，确保新旧知识的有效融合与迭代更新。实验表明，该方法在知识保留与推理准确性方面优于现有主流对比模型。
原文摘要 · Abstract (English)
In the contemporary context of rapid advancements in information technology and the exponential growth of data volume, language models are confronted with significant challenges in effectively navigating the dynamic and ever-evolving information landscape to update and adapt to novel knowledge in real time. In this work, an online update method is proposed, which is based on the existing Retrieval Enhanced Generation (RAG) model with multiple innovation mechanisms. Firstly, the dynamic memory is used to capture the emerging data samples, and then gradually integrate them into the core model through a tunable knowledge distillation strategy. At the same time, hierarchical indexing and multi-layer gating mechanism are introduced into the retrieval module to ensure that the retrieved content is more targeted and accurate. Finally, a multi-stage network structure is established for different types of inputs in the generation stage, and cross-attention matching and screening are carried out on the intermediate representations of each stage to ensure the effective integration and iterative update of new and old knowledge. Experimental results show that the proposed method is better than the existing mainstream comparison models in terms of knowledge retention and inference accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。