arXiv:2502.20245cs.CL2025-02被引 5

对比检索、生成与混合模型在开放域问答中的表现差异。

From Retrieval to Generation: Comparing Different Approaches

  • 系统评估检索、生成与混合模型在知识密集型任务中的表现。
  • DPR在NQ数据集上达50.17%的Top-1准确率,混合模型使BEIR的nDCG@10提升至52.59。
  • 检索方法在生成任务中更低困惑度,适合增强语言模型的准确性。

知识密集型任务如开放域问答(ODQA)、文档重排序及检索增强语言建模,需在检索精度与生成灵活性间取得平衡。传统检索模型如BM25和稠密段落检索(DPR)虽能高效从大规模语料库中检索,但语义深度不足;生成模型如GPT-4-o具备更强上下文理解能力,却难以保证事实一致性。本文系统评估了基于检索、生成及混合模型的表现,重点关注其在ODQA与相关检索增强任务中的性能。结果表明,稠密检索器(尤其是DPR)在NQ数据集上达到50.17%的Top-1准确率;混合模型将BEIR数据集的nDCG@10从BM25的43.42提升至52.59,显示其在文档重排序中的优势。此外,在WikiText-103语言建模任务中,检索方法(如BM25)的困惑度低于生成与混合方法,凸显其在检索增强生成中的实用性。研究为未来优化检索、重排序与生成模型提供了具体依据。

原文摘要 · Abstract (English)

Knowledge-intensive tasks, particularly open-domain question answering (ODQA), document reranking, and retrieval-augmented language modeling, require a balance between retrieval accuracy and generative flexibility. Traditional retrieval models such as BM25 and Dense Passage Retrieval (DPR), efficiently retrieve from large corpora but often lack semantic depth. Generative models like GPT-4-o provide richer contextual understanding but face challenges in maintaining factual consistency. In this work, we conduct a systematic evaluation of retrieval-based, generation-based, and hybrid models, with a primary focus on their performance in ODQA and related retrieval-augmented tasks. Our results show that dense retrievers, particularly DPR, achieve strong performance in ODQA with a top-1 accuracy of 50.17\% on NQ, while hybrid models improve nDCG@10 scores on BEIR from 43.42 (BM25) to 52.59, demonstrating their strength in document reranking. Additionally, we analyze language modeling tasks using WikiText-103, showing that retrieval-based approaches like BM25 achieve lower perplexity compared to generative and hybrid methods, highlighting their utility in retrieval-augmented generation. By providing detailed comparisons and practical insights into the conditions where each approach excels, we aim to facilitate future optimizations in retrieval, reranking, and generative models for ODQA and related knowledge-intensive applications.

开放域问答检索增强模型对比

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。