arXiv:2502.11228cs.CLcs.AI2025-02被引 22

通过动态平衡检索多样性与答案质量,显著提升多跳问答准确率。

Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMs

  • 迭代优化检索多样性与答案质量,利用评分机制动态调整文档选择。
  • 在三个数据集上最高提升4.2%准确率,尤其在多文档场景下优势更明显。
  • 适用于各类大模型,不依赖特定模型,适合复杂推理任务的增强系统。

检索增强生成(RAG)通过引入外部知识源提升大语言模型在领域特定问答任务中的表现。然而,传统RAG系统主要关注相关性检索,常因信息冗余而影响多源信息融合的推理效果。本文提出Vendi-RAG框架,基于迭代过程联合优化检索多样性与答案质量。该框架采用灵活的语义多样性度量——Vendi Score(VS),促进文档检索的语义多样性;并引入大模型判断器,在推理后评估候选答案并输出评分,供检索器在每轮迭代中权衡相关性与多样性。在HotpotQA、MuSiQue和2WikiMultiHopQA三个挑战性数据集上的实验表明,Vendi-RAG在多跳推理任务中显著优于传统单步与多步RAG方法,相比当前最优基线Adaptive-RAG,准确率提升达+4.2%(HotpotQA)、+4.1%(2WikiMultiHopQA)和+1.3%(MuSiQue)。随着检索文档数增加,性能提升更加明显。此外,该框架在GPT-3.5、GPT-4和GPT-4o-mini等多种模型上均表现出一致改进,证明其模型无关性。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) enhances large language models (LLMs) for domain-specific question-answering (QA) tasks by leveraging external knowledge sources. However, traditional RAG systems primarily focus on relevance-based retrieval and often struggle with redundancy, especially when reasoning requires connecting information from multiple sources. This paper introduces Vendi-RAG, a framework based on an iterative process that jointly optimizes retrieval diversity and answer quality. This joint optimization leads to significantly higher accuracy for multi-hop QA tasks. Vendi-RAG leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to promote semantic diversity in document retrieval. It then uses an LLM judge that evaluates candidate answers, generated after a reasoning step, and outputs a score that the retriever uses to balance relevance and diversity among the retrieved documents during each iteration. Experiments on three challenging datasets -- HotpotQA, MuSiQue, and 2WikiMultiHopQA -- demonstrate Vendi-RAG's effectiveness in multi-hop reasoning tasks. The framework achieves significant accuracy improvements over traditional single-step and multi-step RAG approaches, with accuracy increases reaching up to +4.2% on HotpotQA, +4.1% on 2WikiMultiHopQA, and +1.3% on MuSiQue compared to Adaptive-RAG, the current best baseline. The benefits of Vendi-RAG are even more pronounced as the number of retrieved documents increases. Finally, we evaluated Vendi-RAG across different LLM backbones, including GPT-3.5, GPT-4, and GPT-4o-mini, and observed consistent improvements, demonstrating that the framework's advantages are model-agnostic.

RAG多跳推理大模型检索优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。