用影响值筛选检索上下文,让大模型生成更准。
Influence Guided Context Selection for Effective Retrieval-Augmented Generation
- 提出CI值衡量上下文重要性,融合查询、列表和生成器信息。
- 在8个任务上显著优于现有方法,过滤噪声上下文同时保留关键信息。
- 无需调参,只需保留正CI值上下文,适合实际部署使用。
检索增强生成(RAG)通过引入外部知识缓解大语言模型的幻觉问题,但其效果常受低质量检索上下文(含无关或噪声信息)拖累。现有方法依赖预定义的质量评估指标进行上下文选择,提升有限。我们发现根本原因在于未能全面利用查询、上下文列表和生成器的信息进行综合质量评估。受数据选择研究启发,我们将上下文质量评估重新定义为推理时的数据价值评估问题,提出上下文影响值(CI value)。该指标通过移除每个上下文后性能下降程度来量化其质量,有效整合了查询感知的相关性、列表感知的独特性以及生成器感知的一致性。此外,CI value无需复杂超参数调优,仅需保留正值上下文即可。为解决标签依赖与计算开销问题,我们设计了可参数化的代理模型,采用分层架构捕捉局部查询-上下文相关性和全局上下文间交互,通过真值CI值监督和端到端生成反馈训练。在8个NLP任务和多个LLM上的实验表明,该方法显著优于当前最佳基线,能有效过滤劣质上下文并保留关键信息。代码已公开于https://github.com/SJTU-DMTai/RAG-CSM。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) addresses large language model (LLM) hallucinations by grounding responses in external knowledge, but its effectiveness is compromised by poor-quality retrieved contexts containing irrelevant or noisy information. While existing approaches attempt to improve performance through context selection based on predefined context quality assessment metrics, they show limited gains over standard RAG. We attribute this limitation to their failure in holistically utilizing available information (query, context list, and generator) for comprehensive quality assessment. Inspired by recent advances in data selection, we reconceptualize context quality assessment as an inference-time data valuation problem and introduce the Contextual Influence Value (CI value). This novel metric quantifies context quality by measuring the performance degradation when removing each context from the list, effectively integrating query-aware relevance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value eliminates complex selection hyperparameter tuning by simply retaining contexts with positive CI values. To address practical challenges of label dependency and computational overhead, we develop a parameterized surrogate model for CI value prediction during inference. The model employs a hierarchical architecture that captures both local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback. Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that our context selection method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information. Code is available at https://github.com/SJTU-DMTai/RAG-CSM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。