研究检索模型对信息位置的偏倚，发现后段相关文本易被忽略。
An Empirical Study of Position Bias in Modern Information Retrieval
- 构建新评测框架与位置敏感度指标，量化模型对位置的依赖
- 密集嵌入模型在关键信息靠后时性能平均下降15.6%
- 传统BM25和重排序模型更抗位置变化，适合实际应用
本研究探究信息检索中的位置偏倚问题，即模型倾向于过度关注段落开头内容，而忽略后续语义相关的信息。为此，我们提出一个包含两个位置感知检索基准（SQuAD-PosQ、FineWeb-PosQ）和直观诊断指标——位置敏感度指数（PSI）的新评估框架，从最坏情况量化位置偏倚。我们在完整检索流程中进行了全面评估，涵盖BM25、密集嵌入模型、ColBERT类晚期交互模型以及全交互重排序模型。实验表明，当相关信息出现在段落后部时，密集嵌入模型和ColBERT类模型性能显著下降，平均降幅达15.6%；相比之下，BM25与重排序模型表现出更强的鲁棒性。这些发现揭示了模型对相关信息位置的敏感性，为构建更抗位置偏倚的检索系统提供了实践指导。代码与数据公开于：https://github.com/NovaSearch-Team/position-bias-in-IR。
原文摘要 · Abstract (English)
This study investigates the position bias in information retrieval, where models tend to overemphasize content at the beginning of passages while neglecting semantically relevant information that appears later. To analyze the extent and impact of position bias, we introduce a new evaluation framework consisting of two position-aware retrieval benchmarks (SQuAD-PosQ, FineWeb-PosQ) and an intuitive diagnostic metric, the Position Sensitivity Index (PSI), for quantifying position bias from a worst-case perspective. We conduct a comprehensive evaluation across the full retrieval pipeline, including BM25, dense embedding models, ColBERT-style late-interaction models, and full-interaction reranker models. Our experiments show that when relevant information appears later in the passage, dense embedding models and ColBERT-style models suffer significant performance degradation (an average drop of 15.6%). In contrast, BM25 and reranker models demonstrate greater robustness to such positional variation. These findings provide practical insights into model sensitivity to the position of relevant information and offer guidance for building more position-robust retrieval systems. Code and data are publicly available at: https://github.com/NovaSearch-Team/position-bias-in-IR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。