发现并修复检索系统的盲区,提升问答准确率。
With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spots
- 通过嵌入空间几何分析预测检索盲点
- 新方法使检索效果平均提升3.4~4.5分
- 适合追求高可靠性的大模型系统开发者
可靠的检索增强生成(RAG)系统依赖于检索器发现相关资讯的能力。本文揭示当前神经检索器存在盲点——即无法检索到与查询相关但嵌入相似度低的实体。这些盲点源于训练带来的偏差,导致相关实体被映射至嵌入空间中难以触及的位置,从而降低可检索性。我们基于维基数据关系和维基百科首段构建大规模数据集,并提出检索概率评分(RPS),证明标准检索器(如CONTRIEVER、REASONIR)的盲点风险可在索引前通过实体嵌入几何特征预测,无需昂贵的检索评估。为解决此问题,我们提出ARGUS管道,通过从知识库(如维基百科首段)针对性扩充文档,提升高风险(低RPS)实体的可检索性。在BRIGHT、IMPLIRET和RAR-B上的实验表明,ARGUS对所有评估检索器均带来持续改进,平均提升nDCG@5达+3.4分,nDCG@10达+4.5分,尤其在挑战性子集上增益更显著。结果表明,提前修复盲点对构建稳健可信的RAG系统至关重要。
原文摘要 · Abstract (English)
Reliable retrieval-augmented generation (RAG) systems depend fundamentally on the retriever's ability to find relevant information. We show that neural retrievers used in RAG systems have blind spots, which we define as the failure to retrieve entities that are relevant to the query, but have low similarity to the query embedding. We investigate the training-induced biases that cause such blind spot entities to be mapped to inaccessible parts of the embedding space, resulting in low retrievability. Using a large-scale dataset constructed from Wikidata relations and first paragraphs of Wikipedia, and our proposed Retrieval Probability Score (RPS), we show that blind spot risk in standard retrievers (e.g., CONTRIEVER, REASONIR) can be predicted pre-index from entity embedding geometry, avoiding expensive retrieval evaluations. To address these blind spots, we introduce ARGUS, a pipeline that enables the retrievability of high-risk (low-RPS) entities through targeted document augmentation from a knowledge base (KB), first paragraphs of Wikipedia, in our case. Extensive experiments on BRIGHT, IMPLIRET, and RAR-B show that ARGUS achieves consistent improvements across all evaluated retrievers (averaging +3.4 nDCG@5 and +4.5 nDCG@10 absolute points), with substantially larger gains in challenging subsets. These results establish that preemptively remedying blind spots is critical for building robust and trustworthy RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。