让检索模型对语义相似的查询返回一致结果，提升可靠性。
Improving Document Retrieval Coherence for Semantically Equivalent Queries
- 用改进的多负样本损失函数，让模型对相似问题返回相近文档。
- 在多个数据集上，模型敏感度降低，准确率反而提升。
- 适合需要稳定检索结果的问答与信息定位场景。
密集检索（Dense Retrieval, DR）模型在文档检索与信息定位任务中表现优异。通常这类模型训练目标是提升给定查询下前k个相关文档的相关性。已有研究表明，主流DR模型对查询和文档词汇敏感：微小词汇变化可能导致检索结果显著差异。本文提出一种改进的多负样本排序损失函数，用于训练DR模型，以增强其对语义等价查询的检索一致性。该损失函数惩罚不同但语义等价查询所返回的前k个文档之间的差异。我们在MS-MARCO、Natural Questions、BEIR以及TREC DL 19/20等多个数据集上进行了广泛实验。结果表明：(i) 使用该损失训练的模型对输入变化更不敏感；(ii) 有趣的是，模型准确率也更高。
原文摘要 · Abstract (English)
Dense Retrieval (DR) models have proven to be effective for Document Retrieval and Information Grounding tasks. Usually, these models are trained and optimized for improving the relevance of top-ranked documents for a given query. Previous work has shown that popular DR models are sensitive to the query and document lexicon: small variations of it may lead to a significant difference in the set of retrieved documents. In this paper, we propose a variation of the Multi-Negative Ranking loss for training DR that improves the coherence of models in retrieving the same documents with respect to semantically similar queries. The loss penalizes discrepancies between the top-k ranked documents retrieved for diverse but semantic equivalent queries. We conducted extensive experiments on various datasets, MS-MARCO, Natural Questions, BEIR, and TREC DL 19/20. The results show that (i) models optimizes by our loss are subject to lower sensitivity, and, (ii) interestingly, higher accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。