提出量化RAG中检索与生成对齐度的新方法，提升系统可解释性。
Quantifying Retriever-Generator Alignment in RAG with Local Explanations
- 用积分梯度和蒙特卡洛近似沙普利值分析检索与生成组件
- 发现生成器常忽略高排名文档，依赖相关性较低的文档
- 引入WARG指标评估对齐度，适合高风险场景的RAG系统审计
检索增强生成（RAG）系统通过密集检索器与语言模型结合，将输出锚定在外部文档上。然而，两组件间的交互过程仍不透明，给高风险领域部署带来挑战。本文提出RAG-E，一个端到端可解释性框架，通过数学基础的归因方法量化检索器与生成器的对齐程度。该方法将积分梯度拓展至检索器分析，提出基于蒙特卡洛稳定的沙普利值近似用于生成器归因，并引入加权归因-相关性差距（WARG）指标，衡量生成器使用文档与检索器排序的一致性。在PopQA、QAMPARI和TREC CAST数据集上的实验表明存在显著错位：根据模型与设置不同，生成器常忽略最高排名文档，转而依赖相关性较低的文档。结果显示，WARG比皮尔逊和斯皮尔曼相关性更能准确捕捉对齐度，且可作为RAG性能的有效指示器。RAG-E与WARG为审计该交互提供了实用框架，助力构建更可靠、透明的RAG系统。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems combine dense retrievers and language models to ground their outputs in external documents. However, the interaction between these components remains opaque, creating challenges for deployment in high-stakes domains. We present RAG-E, an end-to-end explainability framework that quantifies retriever-generator alignment through mathematically grounded attribution methods. Our approach adapts Integrated Gradients for retriever analysis, proposes a Monte Carlo-stabilized Shapley Value approximation for generator attribution, and introduces the Weighted Attribution-Relevance Gap (WARG) metric to measure how closely the generator's document usage aligns with retriever rankings. Experiments on PopQA, QAMPARI, and TREC CAST datasets reveal substantial misalignment: depending on the model and setting, generators often ignore top-ranked documents and rely on documents ranked as less relevant. We show that WARG captures retriever-generator alignment better than Pearson and Spearman correlations and can serve as an indicator of RAG performance. RAG-E and WARG provide a practical framework for auditing this interaction, enabling more reliable and transparent RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。