用词义关联库揭示检索模型的全局偏见
Discovering Biases in Information Retrieval Models Using Relevance Thesaurus as Global Explanation
- 构建语义相关词对库，解释神经检索模型整体行为
- 发现模型存在品牌名称偏好，影响排序公平性
- 适合关注模型可解释性与公平性的研究者
现有神经相关性模型的解释多聚焦局部，难以预测模型在未见查询-文档对上的表现。本文提出一种新方法，通过构建包含语义相关查询与文档词对的「相关性词典」，全局解释神经相关性模型。该词典用于增强如BM25等词法匹配模型，以逼近神经模型的预测结果。方法先训练神经模型对部分查询和文档片段进行相关性评分，再据此识别词汇空间中的相关词对。评估表明，该词典在排名效果和对目标神经模型的保真度上表现良好。值得注意的是，词典揭示了模型存在品牌名称偏见，凸显了该方法的优势。
原文摘要 · Abstract (English)
Most efforts in interpreting neural relevance models have focused on local explanations, which explain the relevance of a document to a query but are not useful in predicting the model's behavior on unseen query-document pairs. We propose a novel method to globally explain neural relevance models by constructing a "relevance thesaurus" containing semantically relevant query and document term pairs. This thesaurus is used to augment lexical matching models such as BM25 to approximate the neural model's predictions. Our method involves training a neural relevance model to score the relevance of partial query and document segments, which is then used to identify relevant terms across the vocabulary space. We evaluate the obtained thesaurus explanation based on ranking effectiveness and fidelity to the target neural ranking model. Notably, our thesaurus reveals the existence of brand name bias in ranking models, demonstrating one advantage of our explanation method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。