让检索器学会选对任务有帮助的文本,提升大模型表现。
Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models
- 构建共享上下文,让检索器学习跨任务的实用信息。
- 通过扰动归因法评估段落价值,反馈更精准。
- 在10个数据集上验证,适配多种任务场景。
检索增强语言模型通过外部知识检索提升任务性能,但现有检索器主要关注语义相关性,未必适合生成任务。为此,基于效用的检索成为新方向,强调选择对下游任务真正有益的文本。然而,准确捕捉段落效用仍缺乏有效方法。本文提出SCARLet框架,训练效用导向的检索器,融合多任务泛化与段落间交互机制。首先,构造共享上下文,合成多任务训练数据,缓解上下文差异带来的语义偏差,使检索器专注学习任务特定效用并实现跨任务泛化。其次,采用基于扰动的归因方法,估计共享上下文中各段落的效用,反映段落间相互作用,提供更准确的反馈信号。在涵盖多个任务的10个数据集(含域内与域外)上评估表明,使用SCARLet训练的检索器能持续提升RALMs的整体性能。
原文摘要 · Abstract (English)
Retrieval-Augmented Language Models boost task performance, owing to the retriever that provides external knowledge. Although crucial, the retriever primarily focuses on semantics relevance, which may not always be effective for generation. Thus, utility-based retrieval has emerged as a promising topic, prioritizing passages that provide valid benefits for downstream tasks. However, due to insufficient understanding, capturing passage utility accurately remains unexplored. This work proposes SCARLet, a framework for training utility-based retrievers in RALMs, which incorporates two key factors, multi-task generalization and inter-passage interaction. First, SCARLet constructs shared context on which training data for various tasks is synthesized. This mitigates semantic bias from context differences, allowing retrievers to focus on learning task-specific utility and generalize across tasks. Next, SCARLet uses a perturbation-based attribution method to estimate passage-level utility for shared context, which reflects interactions between passages and provides more accurate feedback. We evaluate our approach on ten datasets across various tasks, both in-domain and out-of-domain, showing that retrievers trained by SCARLet consistently improve the overall performance of RALMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。