通过指令驱动减少重复检索内容,显著提升自适应RAG的生成效率。
Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlaps
- 用指令引导模型专注关键内容,避免重复计算冗余信息。
- 预填充阶段加速2.79倍,解码阶段加速2.33倍,质量不变。
- 适用于各类自适应RAG系统,无需修改模型结构。
检索增强生成(RAG)已成为扩展大语言模型知识的关键方法。为更好应对复杂查询,研究者提出自适应RAG(A-RAG),通过多次与外部知识库交互提升生成质量。然而,A-RAG加剧了原有RAG的效率瓶颈,因其依赖多轮生成。现有方法对每轮检索内容均从头处理,忽视多轮间内容高度重叠的问题。重复内容导致大量冗余表示与计算,影响整体效率。本文提出一种模型无关的方法,可通用应用于各类A-RAG系统,旨在减少因检索结果重叠带来的冗余表示。具体而言,采用缓存访问加速预填充阶段,使用并行生成加速解码阶段;同时引入指令驱动模块,引导模型更有效地关注各部分内容,更适合大语言模型处理。实验表明,本方法在预填充和解码阶段分别实现平均2.79倍和2.33倍的加速,且生成质量保持一致。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) has emerged as a pivotal method for expanding the knowledge of large language models. To handle complex queries more effectively, researchers developed Adaptive-RAG (A-RAG) to enhance the generated quality through multiple interactions with external knowledge bases. Despite its effectiveness, A-RAG exacerbates the pre-existing efficiency challenges inherent in RAG, which are attributable to its reliance on multiple iterations of generation. Existing A-RAG approaches process all retrieved contents from scratch. However, they ignore the situation where there is a significant overlap in the content of the retrieval results across rounds. The overlapping content is redundantly represented, which leads to a large proportion of repeated computations, thus affecting the overall efficiency. To address this issue, this paper introduces a model-agnostic approach that can be generally applied to A-RAG methods, which is dedicated to reducing the redundant representation process caused by the overlapping of retrieval results. Specifically, we use cache access and parallel generation to speed up the prefilling and decoding stages respectively. Additionally, we also propose an instruction-driven module to further guide the model to more effectively attend to each part of the content in a more suitable way for LLMs. Experiments show that our approach achieves 2.79 and 2.33 times significant acceleration on average for prefilling and decoding respectively while maintaining equal generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。