arXiv:2601.11273cs.IR2026-01被引 4

让文档排序更懂生成器偏好,提升问答准确率

Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking

  • 基于生成器感知的文档选择与排序联合优化
  • 在5个基准上平均提升F1达2.08,最高增益+2.08
  • 适配不同生成器,尤其对多样性要求高的场景

在RAG范式中,文档排序决定了下游生成器可利用的证据。通过受控分析,我们发现现有排序器未充分建模两个现象:(i) 下游响应质量不仅依赖相关性,还受所选文档组合与顺序的影响;(ii) 不同生成器对此类偏好存在系统性差异。然而,现有排序器仅基于查询-文档相关性训练,忽略了上述因素。为填补这一空白,我们构建了PRISM——一个双语偏好对齐数据集,通过四阶段流程将组合与排序空间压缩约四个数量级,并生成针对七种下游生成器的响应质量偏好监督信号。在包含13,000个查询的PRISM子集上,我们训练了Rank4Gen,一种生成器感知的排序器,可联合完成文档集选择与排序。在五个挑战性RAG基准上的实验表明,Rank4Gen在多数评估生成器上提升了下游问答质量,相较最强基线最大提升达+2.08 F1。

原文摘要 · Abstract (English)

In the RAG paradigm, document ranking determines the evidence available to downstream generators. Through controlled analysis, we identify two phenomena underexplored by existing rankers: (i) downstream response quality depends not only on relevance but also on the composition and ordering of selected documents, and (ii) such preferences differ systematically across generators. However, existing rankers are trained purely on query--document relevance, leaving both phenomena unmodeled. To close this gap, we construct \textbf{PRISM}, a bilingual preference-aligned dataset built through a four-stage pipeline that compresses the combinatorial subset-and-ordering space by roughly four orders of magnitude and produces response-quality preference supervision conditioned on seven downstream generators. On a 13k-query subset of PRISM, we train \textbf{Rank4Gen}, a generator-aware ranker that performs joint document set selection and ordering. Experiments on five challenging RAG benchmarks show that Rank4Gen improves downstream QA quality on most evaluated generators, with per-generator F1 gains of up to $+2.08$ over the strongest set-selection baseline. Code is available at https://github.com/JOHNNY-fans/Rank4Gen.

RAG文档排序生成器感知偏好学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。