在强重排序器下,多数检索增强方法无效,仅查询扩展和SSCC有效。
Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?
- 引入SSCC,按数据源校准阈值,提升异构数据检索
- 强交叉编码器重排序主导性能,其余方法增益不显著
- 实验基于新基准HetDocQA,覆盖代码、表格等多格式数据
检索增强生成(RAG)常通过查询扩展、分层摘要、图拓展、按查询路由、排序融合和纠正性再检索等方法改进检索效果。这些方法的收益主要来自同质语料库,如维基百科纯文本。但在实际中更常见的异构语料库——包含代码、Markdown、表格、科学PDF与文本混合——其有效性尚未被评估。为此,我们构建了 extbf{HetDocQA},一个具有片段重叠相关性标签且集合互斥划分的异构基准,并与MuSiQue和QASPER作为同质对照。在统一主干模型上评估八种方法,使用自助置信区间和多重比较校正。结果表明:强交叉编码器重排序器已主导整体质量;在此基础上,仅有查询扩展和SSCC带来可靠提升。SSCC是一种本文提出的按源校准纠正器,为每个得分源设定独立接受阈值,仅在异构数据上有效。其余常见方法如分层摘要、图扩展、路由和排序融合,在重排序器存在时均无可靠增益。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) is routinely extended with methods meant to improve retrieval: query expansion, hierarchical and cross-document summarization, graph-based expansion, per-query routing, rank fusion, and corrective re-retrieval. The benefits reported for these methods come almost exclusively from homogeneous corpora, predominantly Wikipedia prose. Whether they hold on the mixed-format collections common in practice, where code, markdown, tables, scientific PDFs, and prose are interleaved within one corpus, has not been measured. To study this directly, we build \textbf{HetDocQA}, a heterogeneous benchmark with \emph{chunker-agnostic} span-overlap relevance labels and collection-disjoint splits, and pair it with MuSiQue and QASPER as homogeneous controls. We evaluate eight methods on a shared backbone, with bootstrap confidence intervals and multiple-comparison correction. A strong cross-encoder reranker accounts for most of the pipeline's quality; beyond it, only two methods yield reliable gains: query expansion and SSCC. SSCC, a per-source calibrated corrector introduced here, sets a separate acceptance threshold for each score source and helps only on heterogeneous data. The remaining reranking and pool-expansion methods in common use, among them hierarchical summarization, graph expansion, routing, and rank fusion, give no reliable gain once that reranker is present.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。