通过优化查询提升大模型检索精度，让答案更准更省资源。
Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models
- 先提取大模型参数化知识，再用可训练小模型优化查询。
- 在多个问答数据集上显著优于现有方法，提升回答准确率。
- 适合追求高精度与低计算成本的RAG系统应用者。
我们提出一种名为提取-精炼-检索-阅读（ERRR）的新框架，旨在通过针对大语言模型（LLM）知识需求定制的查询优化，弥补检索增强生成（RAG）系统中的预检索信息鸿沟。与传统RAG查询优化不同，ERRR首先从LLM中提取参数化知识，随后利用专用查询优化器对查询进行精炼，确保仅检索生成准确回答所必需的信息。此外，为提升灵活性并降低计算开销，我们设计了一种可训练方案：采用较小的可调模型作为查询优化器，并通过知识蒸馏从更大的教师模型中学习。在多个问答（QA）数据集及不同检索系统上的评估表明，ERRR持续优于现有基线，证明其是提升RAG系统实用性和准确性的通用且低成本模块。
原文摘要 · Abstract (English)
We introduce the \textit{Extract-Refine-Retrieve-Read} (ERRR) framework, a novel approach designed to bridge the pre-retrieval information gap in Retrieval-Augmented Generation (RAG) systems through query optimization tailored to meet the specific knowledge requirements of Large Language Models (LLMs). Unlike conventional query optimization techniques used in RAG, the ERRR framework begins by extracting parametric knowledge from LLMs, followed by using a specialized query optimizer for refining these queries. This process ensures the retrieval of only the most pertinent information essential for generating accurate responses. Moreover, to enhance flexibility and reduce computational costs, we propose a trainable scheme for our pipeline that utilizes a smaller, tunable model as the query optimizer, which is refined through knowledge distillation from a larger teacher model. Our evaluations on various question-answering (QA) datasets and with different retrieval systems show that ERRR consistently outperforms existing baselines, proving to be a versatile and cost-effective module for improving the utility and accuracy of RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。