对比长上下文与检索增强生成，发现前者在问答中更优，后者对话场景更有优势。
Long Context vs. RAG for LLMs: An Evaluation and Revisits
- 通过筛选无需外部信息的问题，更客观评估长上下文与检索增强的效果。
- 长上下文在维基百科问答中表现优于检索增强，摘要式检索接近长上下文效果。
- 检索增强在对话和通用问题上更具优势，提示需重视上下文相关性。
扩展上下文窗口（长上下文，LC）和使用检索器选择性获取相关信息（检索增强生成，RAG）是让大模型利用超长外部上下文的两种主要策略。本文重访近期相关研究，揭示其核心见解与分歧。通过剔除无需外部信息即可回答的问题、识别最有效的检索方法并扩展数据集，我们发现LC在问答基准测试中普遍优于RAG，尤其在基于维基百科的问题上表现突出。基于摘要的检索表现接近LC，而基于分块的检索则明显落后。然而，RAG在对话类和通用问题查询中具有优势。这些发现凸显了RAG与LC之间的权衡，为未来大模型外源知识优化提供指导。本文还深入讨论该主题，强调现有研究中对上下文相关性的忽视。
原文摘要 · Abstract (English)
Extending context windows (i.e., Long Context, LC) and using retrievers to selectively access relevant information (i.e., Retrieval-Augmented Generation, RAG) are the two main strategies to enable LLMs to incorporate extremely long external contexts. This paper revisits recent studies on this topic, highlighting their key insights and discrepancies. We then provide a more comprehensive evaluation by filtering out questions answerable without external context, identifying the most effective retrieval methods, and expanding the datasets. We show that LC generally outperforms RAG in question-answering benchmarks, especially for Wikipedia-based questions. Summarization-based retrieval performs comparably to LC, while chunk-based retrieval lags behind. However, RAG has advantages in dialogue-based and general question queries. These insights underscore the trade-offs between RAG and LC strategies, offering guidance for future optimization of LLMs with external knowledge sources. We also provide an in-depth discussion on this topic, highlighting the overlooked importance of context relevance in existing studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。