用成对相关性蒸馏提升密集检索效果，更精准区分相似文档。
PairDistill: Pairwise Relevance Distillation for Dense Retrieval
- 基于成对排序信息进行知识蒸馏，替代传统点对点评分
- 在多个基准上达到新最优性能，显著提升检索精度
- 适合需要高精度文档排序的检索系统研发人员
从海量数据中高效获取相关信息依赖先进的检索技术。与传统稀疏检索相比，密集检索近年来展现出显著优势。为进一步提升检索性能，研究者广泛探索了利用强效交叉编码器重排器的知识蒸馏方法。然而，现有方法多基于点对点重排器，仅给出文档的绝对相关性分数，导致比较不一致。本文提出成对相关性蒸馏（PairDistill），利用成对重排信息，实现对相似相关文档的细粒度区分，从而增强密集检索模型的训练。实验表明，PairDistill在多个基准上超越现有方法，达到新的最优水平，凸显其在提升密集检索技术方面的潜力。源代码与训练模型已公开于 https://github.com/MiuLab/PairDistill。
原文摘要 · Abstract (English)
Effective information retrieval (IR) from vast datasets relies on advanced techniques to extract relevant information in response to queries. Recent advancements in dense retrieval have showcased remarkable efficacy compared to traditional sparse retrieval methods. To further enhance retrieval performance, knowledge distillation techniques, often leveraging robust cross-encoder rerankers, have been extensively explored. However, existing approaches primarily distill knowledge from pointwise rerankers, which assign absolute relevance scores to documents, thus facing challenges related to inconsistent comparisons. This paper introduces Pairwise Relevance Distillation (PairDistill) to leverage pairwise reranking, offering fine-grained distinctions between similarly relevant documents to enrich the training of dense retrieval models. Our experiments demonstrate that PairDistill outperforms existing methods, achieving new state-of-the-art results across multiple benchmarks. This highlights the potential of PairDistill in advancing dense retrieval techniques effectively. Our source code and trained models are released at https://github.com/MiuLab/PairDistill
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。