增加文档数量会显著降低RAG模型表现，但对某些模型影响较小。
More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAG
- 控制上下文长度不变，只改变文档数量，隔离多文档挑战。
- 多数LLM在文档增多时性能下降最高达20%。
- Qwen2.5表现稳定，适合需要处理多文档的场景。
检索增强生成（RAG）通过在生成过程中引入外部文档来提升大语言模型（LLM）的准确性。尽管先前研究指出检索大量文档会降低性能，但未在控制上下文长度的前提下分离文档数量的影响。本文在自定义的多跳问答数据集上评估多种语言模型，保持上下文长度和相关信息位置恒定，仅改变文档数量。结果发现，在RAG设置中增加文档数量对大多数LLM构成显著挑战，性能最高下降20%。然而，Qwen2.5在文档数量增加时表现保持稳定，显示出更强的多文档处理能力。研究还表明，处理多文档是独立于长上下文处理的挑战。相关数据集与代码已开源：https://github.com/shaharl6000/MoreDocsSameLen。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enhances the accuracy of Large Language Model (LLM) responses by leveraging relevant external documents during generation. Although previous studies noted that retrieving many documents can degrade performance, they did not isolate how the quantity of documents affects performance while controlling for context length. We evaluate various language models on custom datasets derived from a multi-hop QA task. We keep the context length and position of relevant information constant while varying the number of documents, and find that increasing the document count in RAG settings poses significant challenges for most LLMs, reducing performance by up to 20%. However, Qwen2.5 maintained consistent results across increasing document counts, indicating better multi-document handling capability. Finally, our results indicate that processing multiple documents is a separate challenge from handling long contexts. We also make the datasets and code available: https://github.com/shaharl6000/MoreDocsSameLen .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。