arXiv:2608.01630cs.CLcs.AI2026-08

让大模型自己检索知识,不用外部检索器也能又准又快。

RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection

论文配图:RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
图 1 · 摘自论文原文
  • 用强化学习训练模型内部记忆的搜索策略
  • 在新数据集上准确率超传统检索和参数注入方法
  • 适合需要持续更新知识的实用场景

检索增强生成(RAG)虽提升事实准确性,但带来延迟和部署复杂性。本文提出 RING(检索内化生成),一种融合架构与训练的完整范式,将大规模外部知识注入到‘记忆专家混合模型’中,并通过强化学习学习参数化检索策略,彻底移除外部检索器。训练分三阶段:持续预训练通过新型双因果注意力将新语料注入知识专家;监督微调教会‘搜索-回答’模式;强化学习结合层次奖励优化路由与搜索策略。不同于以往固定或规则检索的参数化注入方法,RING 直接从任务信号中学习检索策略。理论上,RING 可视为经典 RAG 目标的无检索近似。为评估真正新知识的大规模注入且无测试泄漏,我们构建了新闻-2025(News-2025)基准,数据严格晚于基础大模型预训练截止时间。实验表明,RING 在准确率与效率上均达到或超越基于搜索的 RAG 和参数化注入基线。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) improves factuality but adds latency and engineering overhead at serving time. We propose RING (Retrieval-Internalized Generation), a holistic paradigm spanning both architecture and training that injects large-scale external knowledge into a \textit{Mixture-of-Memory Experts} and learns parametric search over this internal memory via reinforcement learning, removing the external retriever entirely. Training proceeds in three stages: continued pre-training injects new corpora into a Knowledge Expert via our novel \textit{Dual Causal Attention}; supervised fine-tuning teaches a ``search-then-answer'' pattern; and reinforcement learning with hierarchical rewards optimizes the routing-and-search policy over the parametric memory. Unlike prior parametric injection methods that pair internal memory with a fixed or rule-based retriever, RING {learns} its retrieval policy directly from task signals. We further frame RING theoretically as a search-free approximation to the classical RAG objective. To evaluate large-scale injection of genuinely {new} knowledge without test-time leakage, we further construct News-2025, a benchmark built from news strictly post-dating the base LLM's pretraining cutoff. RING matches or surpasses both search-based RAG and parametric injection baselines in accuracy and efficiency.

知识注入大模型强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。