长上下文大模型能提升检索增强生成效果吗？
Long Context RAG Performance of Large Language Models
- 测试20个主流模型在2k-128k tokens上下文下的RAG表现
- 超过64k token时仅少数前沿模型保持稳定准确率
- 发现长上下文下新故障模式，为后续研究指明方向
检索增强生成（RAG）已成为通过引入外部信息提升大语言模型（LLM）准确性的重要技术。随着支持更长上下文长度的LLM出现，人们愈发关注这些模型在RAG场景中的表现。本文对20个主流开源与商用LLM在长上下文条件下的RAG性能进行了全面研究。我们在三个领域特定数据集上运行RAG流程，将总上下文长度从2,000逐步扩展至128,000 tokens（部分模型达2百万tokens），并报告了关于长上下文在RAG中优势与局限的关键洞察。结果表明，尽管增加检索文档数量有助于提升性能，但仅有少数最先进模型能在超过64,000 tokens的长上下文下维持一致的准确性。我们还识别出长上下文场景下的典型失败模式，提示未来研究需重点关注的方向。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) by incorporating external information. With the advent of LLMs that support increasingly longer context lengths, there is a growing interest in understanding how these models perform in RAG scenarios. Can these new long context models improve RAG performance? This paper presents a comprehensive study of the impact of increased context length on RAG performance across 20 popular open source and commercial LLMs. We ran RAG workflows while varying the total context length from 2,000 to 128,000 tokens (and 2 million tokens when possible) on three domain-specific datasets, and report key insights on the benefits and limitations of long context in RAG applications. Our findings reveal that while retrieving more documents can improve performance, only a handful of the most recent state of the art LLMs can maintain consistent accuracy at long context above 64k tokens. We also identify distinct failure modes in long context scenarios, suggesting areas for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。