用可计算的归因方法找出RAG中关键文档,让大模型推理更透明。
Source Attribution in Retrieval-Augmented Generation
- 基于谢帕利值思想设计高效近似方法,降低大模型调用成本。
- 在复杂文档关系下仍能准确识别冗余、互补等关键信息源。
- 适合需要解释性且资源有限的RAG系统开发者参考。
尽管谢帕利值等归因方法在传统机器学习中广泛用于解释特征或训练数据的重要性,但在大语言模型(LLM)尤其是检索增强生成(RAG)系统中的应用仍处于初期阶段且面临挑战。主要障碍是计算成本高昂,每次效用函数评估需调用昂贵的LLM,带来直接的经济和时间开销。本文研究了将基于谢帕利值的归因方法应用于RAG中识别关键检索文档的可行性与有效性。我们对比了谢帕利值与其更易计算的近似方法,以及一些现有的LLM归因方法。研究目标包括:(1)系统性地将成熟的归因原则应用于RAG的文档级场景;(2)量化SHAP近似方法在最小化昂贵的LLM交互前提下,对精确归因的逼近程度;(3)评估其在复杂文档关系(如冗余、互补、协同)下的实际可解释性。本研究旨在弥合强大归因技术与基于LLM的RAG系统实际约束之间的差距,为实现可靠且低成本的RAG可解释性提供洞见。
原文摘要 · Abstract (English)
While attribution methods, such as Shapley values, are widely used to explain the importance of features or training data in traditional machine learning, their application to Large Language Models (LLMs), particularly within Retrieval-Augmented Generation (RAG) systems, is nascent and challenging. The primary obstacle is the substantial computational cost, where each utility function evaluation involves an expensive LLM call, resulting in direct monetary and time expenses. This paper investigates the feasibility and effectiveness of adapting Shapley-based attribution to identify influential retrieved documents in RAG. We compare Shapley with more computationally tractable approximations and some existing attribution methods for LLM. Our work aims to: (1) systematically apply established attribution principles to the RAG document-level setting; (2) quantify how well SHAP approximations can mirror exact attributions while minimizing costly LLM interactions; and (3) evaluate their practical explainability in identifying critical documents, especially under complex inter-document relationships such as redundancy, complementarity, and synergy. This study seeks to bridge the gap between powerful attribution techniques and the practical constraints of LLM-based RAG systems, offering insights into achieving reliable and affordable RAG explainability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。