实验证明：RAG系统中位置偏倚影响有限，因相关与干扰项均被同等削弱。
Do RAG Systems Really Suffer From Positional Bias?
- 通过三组基准测试，分析检索结果位置对LLM生成的影响。
- 超过60%的查询中，前10个召回片段含高度干扰项。
- 重排策略效果不如随机打乱，说明位置偏倚在实际中不显著。
检索增强生成通过将外部语料库中的段落添加到LLM提示中来提升准确性。本文研究了位置偏倚——即LLM根据信息在提示中的位置不同而赋予不同权重的现象——如何影响其利用相关段落的能力，以及对干扰段落的敏感性。在三个基准上的大量实验表明，尽管先进检索管道试图召回相关内容，却系统性地将高度干扰的段落置于前列，超过60%的查询在前10个召回段落中至少包含一个高度干扰项。因此，在真实场景下，尽管在受控环境中位置偏倚常被报告为显著，但实际影响微乎其微，因为相关和干扰段落均被同等削弱。事实上，基于LLM位置偏好进行段落重排的复杂策略，表现并不优于随机打乱。
原文摘要 · Abstract (English)
Retrieval Augmented Generation enhances LLM accuracy by adding passages retrieved from an external corpus to the LLM prompt. This paper investigates how positional bias - the tendency of LLMs to weight information differently based on its position in the prompt - affects not only the LLM's capability to capitalize on relevant passages, but also its susceptibility to distracting passages. Through extensive experiments on three benchmarks, we show how state-of-the-art retrieval pipelines, while attempting to retrieve relevant passages, systematically bring highly distracting ones to the top ranks, with over 60% of queries containing at least one highly distracting passage among the top-10 retrieved passages. As a result, the impact of the LLM positional bias, which in controlled settings is often reported as very prominent by related works, is actually marginal in real scenarios since both relevant and distracting passages are, in turn, penalized. Indeed, our findings reveal that sophisticated strategies that attempt to rearrange the passages based on LLM positional preferences do not perform better than random shuffling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。