用博弈论方法评估上下文重要性,提升RAG系统效率
Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning

- 将上下文视为合作博弈,用Shapley值量化每段信息贡献
- 在长文本问答和多跳推理中达到与主流模型相当的准确率
- 兼顾理论严谨性与实际效率,适合需要可解释性的RAG应用
上下文重排序与剪枝对提升现代检索增强生成(RAG)系统的效率至关重要,但可解释且统一的框架仍不完善。现有方法多依赖词法检索、交叉编码器架构、模型蒸馏和低秩适配(LoRA),主要依靠启发式损失函数和经验归因。本文提出一种名为Shapley Context Pruning(SCP)的新框架,从合作博弈论视角出发,将上下文建模为合作博弈以实现重要性归因。为平衡细粒度与粗粒度表示,采用Deep Sets架构在句子级别近似置换不变的价值函数,使用预训练语言模型作为句子嵌入器,并通过成对边缘排名损失进行优化。为保障实际可扩展性而不牺牲数学严谨性,利用蒙特卡洛采样实现高效训练与推理,提供正式的理论误差界与样本复杂度保证,确保前K项子集排序的稳定性。此外,我们在支持句召回、针堆中找针(NIAH)、长上下文问答和多跳推理等任务上进行了全面实验,并对嵌入质量与归因策略进行了严格的消融研究。模型在下游问答任务中表现优于或媲美多个强基线。
原文摘要 · Abstract (English)
Context reranking and pruning have become essential for improving the efficiency of modern Retrieval-Augmented Generation (RAG) systems, yet an interpretable and unified framework remains underexplored. Previous work has primarily emphasized lexical retrieval, cross-encoder architectures, model distillation, and Low-Rank Adaptation (LoRA), mostly relying on heuristic loss functions and empirical attribution. This paper presents Shapley Context Pruning (SCP), a novel framework for context reranking that establishes a cooperative-game-theory perspective for importance attribution by modeling the context as a cooperative game. Balancing the trade-off between fine-grained and coarse-grained representations, we employ a Deep Sets architecture to approximate a permutation-invariant value function at the sentence level, utilizing pre-trained language models as sentence embedders and optimizing via a pairwise margin ranking loss. To ensure practical scalability without sacrificing mathematical rigor, we leverage Monte-Carlo sampling for efficient training and inference, providing formal theoretical error bounds and sample complexity guarantees for preserving Top-K subset rankings. Furthermore, we conduct comprehensive experiments-spanning supporting-sentence recall, Needle-in-the-Haystack (NIAH) evaluations, long-context QA, and multi-hop reasoning-alongside rigorous ablation studies on embedding quality and attribution strategies. The model achieves competitive downstream QA performance against robust baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。