用对比学习提升文本重排序,让相关段落更突出。
ListConRanker: A Contrastive Text Reranker with Listwise Encoding
- 引入列表级编码,同时比较多个段落与查询的相似性。
- 在多个中文数据集上达到顶尖性能,优于现有方法。
- 采用环形损失函数,训练更稳定高效,适合实际应用。
重排序模型旨在根据查询与段落之间的语义相似性重新排序,近年来因检索增强生成的应用而备受关注。以往方法多采用点对点编码,仅能单独处理每个段落的上下文。但对重排序而言,段落间的相互比较更为关键,即列表级编码。此外,传统模型使用交叉熵损失,导致梯度变化不平滑、训练效率低。为此,我们提出新型列表级对比文本重排序模型(ListConRanker),在编码阶段即引入段落间对比信息,增强正样本间及正负样本间的区分能力。同时采用环形损失训练,提升梯度灵活性,改善训练效率。实验表明,ListConRanker在中文大规模文本嵌入基准的多个重排序数据集(cMedQA1.0、cMedQA2.0、MMarcoReranking、T2Reranking)上均达到当前最优性能。
原文摘要 · Abstract (English)
Reranker models aim to re-rank the passages based on the semantics similarity between the given query and passages, which have recently received more attention due to the wide application of the Retrieval-Augmented Generation. Most previous methods apply pointwise encoding, meaning that it can only encode the context of the query for each passage input into the model. However, for the reranker model, given a query, the comparison results between passages are even more important, which is called listwise encoding. Besides, previous models are trained using the cross-entropy loss function, which leads to issues of unsmooth gradient changes during training and low training efficiency. To address these issues, we propose a novel Listwise-encoded Contrastive text reRanker (ListConRanker). It can help the passage to be compared with other passages during the encoding process, and enhance the contrastive information between positive examples and between positive and negative examples. At the same time, we use the circle loss to train the model to increase the flexibility of gradients and solve the problem of training efficiency. Experimental results show that ListConRanker achieves state-of-the-art performance on the reranking benchmark of Chinese Massive Text Embedding Benchmark, including the cMedQA1.0, cMedQA2.0, MMarcoReranking, and T2Reranking datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。