扩大文档库比升级大模型更能提升RAG效果
Less LLM, More Documents: Searching for Improved RAG
- 通过扩展检索文档库替代增大生成模型规模
- 小中型模型搭配大文档库可达到大型模型性能
- 适合资源有限但需高精度问答的场景
检索增强生成（RAG）结合文档检索与大语言模型。尽管扩大生成器通常能提升准确率，但也带来推理与部署开销。本文研究了另一维度：扩大检索器语料库及其与生成器规模的权衡关系。在多个开放域问答基准上，语料库扩展始终提升RAG表现，许多情况下可媲美换用更大模型带来的收益，但存在规模增大后的边际递减现象。小至中等规模生成器搭配更大语料库常能媲美使用更大模型但语料更小的情况；中等规模模型受益最明显，而极小或极大模型增益较小。分析表明，这些改进主要源于答案相关段落覆盖范围增加，而利用效率基本不变。总体而言，本研究刻画了RAG中语料库与生成器之间的权衡关系，并为二者在实际应用中的协同提供了实证指导。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) couples document retrieval with large language models (LLMs). While scaling generators often improves accuracy, it also increases inference and deployment overhead. We study an orthogonal axis: enlarging the retriever's corpus, and how it trades off with generator scale. Across multiple open-domain QA benchmarks, corpus scaling consistently strengthens RAG and can in many cases match the gains of moving to a larger model tier, though with diminishing returns at larger scales. Small- and mid-sized generators paired with larger corpora often rival much larger models with smaller corpora; mid-sized models tend to gain the most, while tiny and very large models benefit less. Our analysis suggests that these improvements arise primarily from increased coverage of answer-bearing passages, while utilization efficiency remains largely unchanged. Overall, our results characterize a corpus-generator trade-off in RAG and provide empirical guidance on how corpus scale and model capacity interact in this setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。