研究知识筛选如何影响生成效果，发现其作用取决于模型和任务难度。
How Does Knowledge Selection Help Retrieval Augmented Generation?
- 通过控制混合真实与干扰知识，模拟不同检索与筛选条件。
- 强模型在明确任务中提升召回率更重要，弱模型则需依赖筛选优化F1。
- 适合关注RAG系统优化的开发者与研究人员参考。
检索增强生成（RAG）通过将外部知识融入模型输出，显著提升自然语言生成质量。尽管先前研究强调改进知识检索的重要性，但知识筛选（即重排序或过滤）的作用仍不清晰。本文通过控制实验，使用真实知识与干扰知识的混合组合，系统评估了检索与筛选对生成结果的影响。结果表明，下游生成模型的能力、任务复杂度及数据集特性显著影响知识筛选的效果。在典型场景中，提高知识召回率是关键，当使用强生成模型处理清晰任务时，知识筛选带来的收益有限；而在弱生成模型或模糊任务与数据集下，知识F1分数成为核心因素，知识筛选对整体性能提升更为重要。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) is a powerful method for enhancing natural language generation by integrating external knowledge into a model's output. While prior work has demonstrated the importance of improving knowledge retrieval for boosting generation quality, the role of knowledge selection, a.k.a. reranking or filtering, remains less clear. This paper empirically analyzes how knowledge selection influences downstream generation performance in RAG systems. By simulating different retrieval and selection conditions through a controlled mixture of gold and distractor knowledge, we assess the impact of these factors on generation outcomes. Our findings indicate that the downstream generator model's capability, as well as the complexity of the task and dataset, significantly influence the impact of knowledge selection on the overall RAG system performance. In typical scenarios, improving the knowledge recall score is key to enhancing generation outcomes, with the knowledge selector providing limited benefit when a strong generator model is used on clear, well-defined tasks. For weaker generator models or more ambiguous tasks and datasets, the knowledge F1 score becomes a critical factor, and the knowledge selector plays a more prominent role in improving overall performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。