arXiv:2603.11513cs.CL2026-03被引 2

小模型用不好检索信息,反而容易被干扰,导致回答更差。

Can Small Language Models Use What They Retrieve? An Empirical Study of Retrieval Utilization Across Model Scale

  • 通过分离已知与需外部知识的问题,精准定位利用失败
  • 7B以下模型在理想检索下仍错答85%~100%,存在根本性利用瓶颈
  • 引入上下文后,42%~100%的原有正确答案被破坏,易受干扰

检索增强生成(RAG)广泛用于提升语言模型的事实准确性,但小型模型(参数量7B及以下)是否能有效利用检索信息仍不明确。本文评估了从360M到8B的五种模型规模,涵盖SmolLM2、Qwen2.5和Llama 3.1三类架构,在四种检索条件下进行测试:无检索、BM25检索、使用E5-large-v2的稠密检索,以及保证答案存在的“理想检索”(oracle retrieval)。研究提出参数化知识划分方法,将模型可独立回答的问题与需外部知识的问题分离,从而区分利用失败与检索质量不佳。结果发现:第一,即使在理想检索下,7B及以下模型在无法自主回答的问题上仍错误率达85%~100%,表明存在根本性的上下文利用瓶颈;第二,引入检索上下文后,模型原有正确答案被破坏的比例达42%~100%,说明上下文存在即引发干扰,而非其质量所致;第三,对2588个理想检索失败案例的分析显示,主要失败模式为“无关生成”,即模型完全忽略提供内容。这些现象在多种提示模板和检索方法下均成立。结论表明,7B以下模型在RAG中的主要瓶颈是上下文利用能力,而非检索质量,标准评估下部署RAG可能导致整体性能下降。

原文摘要 · Abstract (English)

Retrieval augmented generation RAG is widely deployed to improve factual accuracy in language models yet it remains unclear whether smaller models of size 7B parameters or less can effectively utilize retrieved information. To investigate this question we evaluate five model sizes from 360M to 8B across three architecture families SmolLM2 Qwen2.5 and Llama 3.1 under four retrieval conditions including no retrieval BM25 dense retrieval using E5 large v2 and oracle retrieval where the retrieved passage is guaranteed to contain the answer. We introduce a parametric knowledge split that separates questions a model can already answer from those that require external knowledge which allows us to isolate utilization failure from retrieval quality failure. We find three main results. First even with oracle retrieval models of size 7B or smaller fail to extract the correct answer 85 to 100 percent of the time on questions they cannot answer alone which indicates a fundamental utilization bottleneck. Second adding retrieval context destroys 42 to 100 percent of answers the model previously knew suggesting a distraction effect driven by the presence of context rather than its quality. Third an error analysis of 2588 oracle failures shows that the dominant failure mode is irrelevant generation where the model ignores the provided context entirely. These patterns hold across multiple prompt templates and retrieval methods. The results indicate that for models below 7B parameters the main limitation of RAG is context utilization rather than retrieval quality and that deploying RAG at this scale can lead to a net negative trade off under standard evaluation conditions.

RAG小模型检索利用上下文干扰

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。