用全局对比信息提升零样本文档排序效果,兼顾精度与效率
Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information
- 通过锚文档构建全局对比参考,生成对比性相关得分
- 在TREC DL和BEIR上显著超越现有点式方法,接近对比式方法性能
- 无需训练,可无缝集成到现有方法中,适合高效排序场景
近期研究已成功利用大语言模型实现零样本文档排序,探索多种提示策略。成对和列表式方法虽有效但计算开销大,难以应用于大规模场景;基于打分的点式方法效率高,但因独立评分忽略了文档间的比较信息,导致评分不一致、性能不佳。本文提出一种新的全局一致性对比点式排序(GCCP)策略,通过将每个候选文档与一个锚文档进行全局对比,生成对比性相关得分。该锚文档被设计为伪相关文档的查询聚焦摘要,有效捕捉全局上下文以支持文档间比较。这些对比得分可通过后聚合方式(PAGC)高效整合进现有点式方法,实现无训练的全局上下文融合。在TREC DL和BEIR基准上的实验表明,本方法显著优于以往点式方法,同时保持相近效率,并达到与需大量计算的对比式方法相当的性能。更多分析验证了锚文档构造策略的有效性。
原文摘要 · Abstract (English)
Recent advancements have successfully harnessed the power of Large Language Models (LLMs) for zero-shot document ranking, exploring a variety of prompting strategies. Comparative approaches like pairwise and listwise achieve high effectiveness but are computationally intensive and thus less practical for larger-scale applications. Scoring-based pointwise approaches exhibit superior efficiency by independently and simultaneously generating the relevance scores for each candidate document. However, this independence ignores critical comparative insights between documents, resulting in inconsistent scoring and suboptimal performance. In this paper, we aim to improve the effectiveness of pointwise methods while preserving their efficiency through two key innovations: (1) We propose a novel Global-Consistent Comparative Pointwise Ranking (GCCP) strategy that incorporates global reference comparisons between each candidate and an anchor document to generate contrastive relevance scores. We strategically design the anchor document as a query-focused summary of pseudo-relevant candidates, which serves as an effective reference point by capturing the global context for document comparison. (2) These contrastive relevance scores can be efficiently Post-Aggregated with existing pointwise methods, seamlessly integrating essential Global Context information in a training-free manner (PAGC). Extensive experiments on the TREC DL and BEIR benchmark demonstrate that our approach significantly outperforms previous pointwise methods while maintaining comparable efficiency. Our method also achieves competitive performance against comparative methods that require substantially more computational resources. More analyses further validate the efficacy of our anchor construction strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。