arXiv:2511.00505cs.CL2025-11被引 2

通过去除冗余知识,让大模型更高效地用内建知识回答问题。

Zero-RAG: Towards Retrieval-Augmented Generation with Zero Redundant Knowledge

  • 用新指标识别并移除外部知识库中的冗余内容。
  • 减少30%知识库规模,检索速度提升22%且不影响效果。
  • 适合追求高效、低延迟的RAG应用开发人员。

检索增强生成(RAG)能有效缓解大语言模型的幻觉问题,通常依赖大型外部语料库补充知识。然而随着大模型内部知识的显著扩展,外部语料库与模型内部知识存在大量冗余。这不仅使密集检索的索引成本随语料库增大而上升,且冗余信息反而会降低模型在仅靠自身知识即可回答的问题上的表现。为此,我们提出Zero-RAG,首先引入Mastery-Score指标识别并剔除冗余知识;经剪枝后,对“已掌握”问题的回答主要依赖模型内部知识。为进一步提升内部知识利用效率,设计Query Router与Noise-Tolerant Tuning,避免无关文档干扰。实验表明,Zero-RAG可将Wikipedia语料库缩减30%,检索阶段加速22%,且不损害RAG性能。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation has shown remarkable results to address Large Language Models' hallucinations, which usually uses a large external corpus to supplement knowledge to LLMs. However, with the development of LLMs, the internal knowledge of LLMs has expanded significantly, thus causing significant knowledge redundancy between the external corpus and LLMs. On the one hand, the indexing cost of dense retrieval is highly related to the corpus size and thus significant redundant knowledge intensifies the dense retrieval's workload. On the other hand, the redundant knowledge in the external corpus is not helpful to LLMs and our exploratory analysis shows that it instead hurts the RAG performance on those questions which the LLM can answer by itself. To address these issues, we propose Zero-RAG to tackle these challenges. Specifically, we first propose the Mastery-Score metric to identify redundant knowledge in the RAG corpus to prune it. After pruning, answers to "mastered" questions rely primarily on internal knowledge of the LLM. To better harness the internal capacity, we propose Query Router and Noise-Tolerant Tuning to avoid the irrelevant documents' distraction and thus further improve the LLM's utilization of internal knowledge with pruned corpus. Experimental results show that Zero-RAG prunes the Wikipedia corpus by 30\% and accelerates the retrieval stage by 22\%, without compromising RAG's performance.

RAG知识压缩大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。