研究检索增强生成中上下文大小与模型选择的影响。
On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems
- 测试不同长度上下文片段对问答效果的影响
- 最多15个片段时性能提升，超过则停滞或下降
- 医学领域需专用模型，通用模型在百科类表现更优
检索增强生成（RAG）通过引入动态外部知识，降低大语言模型对静态知识的依赖，提升答案事实性。本文系统评估了不同上下文大小、BM25与语义搜索两种检索方法，以及八种通用大模型在两个领域的长文本问答任务中的表现。不同于以往短答案评估，本研究聚焦更难的长篇问答场景，要求模型完整利用全部检索到的上下文。结果表明，答案质量随上下文数量增加，在最多15个片段时持续提升，超过后趋于停滞甚至下降。此外，不同通用大模型在生物医学与百科类任务中表现差异显著，且在大规模语料库中进行开放域证据检索仍具挑战性。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) has emerged as an approach to augment large language models (LLMs) by reducing their reliance on static knowledge and improving answer factuality. RAG retrieves relevant context snippets and generates an answer based on them. Despite its increasing industrial adoption, systematic exploration of RAG components is lacking, particularly regarding the ideal size of provided context, and the choice of base LLM and retrieval method. To help guide development of robust RAG systems, we evaluate various context sizes, BM25 and semantic search as retrievers, and eight base LLMs. Moving away from the usual RAG evaluation with short answers, we explore the more challenging long-form question answering in two domains, where a good answer has to utilize the entire context. Our findings indicate that final QA performance improves steadily with up to 15 snippets but stagnates or declines beyond that. Finally, we show that different general-purpose LLMs excel in the biomedical domain than the encyclopedic one, and that open-domain evidence retrieval in large corpora is challenging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。