浅层标注数据更利于BERT重排序模型泛化与效果提升
Impact of Shallow vs. Deep Relevance Judgments on BERT-based Reranking Models
- 对比浅层与深层相关性标注对BERT重排序模型的影响
- 浅层数据因上下文丰富，显著提升模型泛化能力
- 适合关注检索模型训练数据设计的研究者
本文研究了浅层与深层相关性标注对基于BERT的重排序模型在神经信息检索中表现的影响。浅层标注数据具有大量查询但每条查询仅有少量相关性判断，而深层标注数据则相反，查询较少但每条查询有大量相关性判断。实验在MS MARCO和LongEval数据集上进行。结果表明，浅层标注数据通常能增强重排序模型的泛化能力与有效性，原因在于其提供了更广泛的上下文。深层标注数据的劣势可通过增加负样本数量部分缓解。
原文摘要 · Abstract (English)
This paper investigates the impact of shallow versus deep relevance judgments on the performance of BERT-based reranking models in neural Information Retrieval. Shallow-judged datasets, characterized by numerous queries each with few relevance judgments, and deep-judged datasets, involving fewer queries with extensive relevance judgments, are compared. The research assesses how these datasets affect the performance of BERT-based reranking models trained on them. The experiments are run on the MS MARCO and LongEval collections. Results indicate that shallow-judged datasets generally enhance generalization and effectiveness of reranking models due to a broader range of available contexts. The disadvantage of the deep-judged datasets might be mitigated by a larger number of negative training examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。