用多样化指令提升大模型的外部知识检索能力
RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions
- 基于五种检索范式合成多样RAG指令数据
- 构建40K规模数据集，覆盖多种任务场景
- 适合需要增强大模型知识推理的研究者
检索增强生成（RAG）已成为通过引入外部知识提升大语言模型（LLM）性能的关键范式。然而现有方法存在两大局限：（1）仅覆盖有限的RAG场景；（2）因缺乏通用RAG数据集导致任务多样性不足。为此，我们提出RAG-Instruct，一种基于任意语料库合成多样化高质量RAG指令数据的通用方法。该方法结合（1）五种RAG范式，涵盖多样的查询-文档关系；（2）指令模拟技术，利用已有指令数据集的优势提升指令多样性和质量。基于此方法，我们在Wikipedia上构建了一个40K条目的指令数据集，全面覆盖多种RAG场景与任务。实验表明，RAG-Instruct能有效提升大模型的RAG能力，在零样本条件下表现优异，并在多个任务上显著优于各类RAG基线方法。代码已开源：https://github.com/FreedomIntelligence/RAG-Instruct。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as a key paradigm for enhancing large language models (LLMs) by incorporating external knowledge. However, current RAG methods face two limitations: (1) they only cover limited RAG scenarios. (2) They suffer from limited task diversity due to the lack of a general RAG dataset. To address these limitations, we propose RAG-Instruct, a general method for synthesizing diverse and high-quality RAG instruction data based on any source corpus. Our approach leverages (1) five RAG paradigms, which encompass diverse query-document relationships, and (2) instruction simulation, which enhances instruction diversity and quality by utilizing the strengths of existing instruction datasets. Using this method, we construct a 40K instruction dataset from Wikipedia, comprehensively covering diverse RAG scenarios and tasks. Experiments demonstrate that RAG-Instruct effectively enhances LLMs' RAG capabilities, achieving strong zero-shot performance and significantly outperforming various RAG baselines across a diverse set of tasks. RAG-Instruct is publicly available at https://github.com/FreedomIntelligence/RAG-Instruct.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。