区分了RAG阅读中竞争性片段与上下文长度的影响。
Separating Semantic Competition from Context Length in RAG Reading
- 设计对照实验，固定片段数量和长度，替换强竞争片段。
- 替换后性能显著回升，Phi-2提升6.0点EM，Qwen2.5提升4.5点EM。
- 揭示竞争效应独立于上下文长度，适合模型评估研究者参考。
检索增强生成（RAG）系统即使在正确段落被检索出时也可能错误回答。模型仍需阅读已检索段落，并从多个看似相关的段落中识别出正确答案所在。这一过程称为‘阅读器’。其失败是因上下文过长，还是因其他段落确实构成竞争？我们提出并验证了一种匹配对照协议：保持段落数量和长度不变，仅将强竞争段落替换为真实但竞争力较弱的段落。该控制在SQuAD上应用于两个小型开源模型。替换后性能部分恢复，尤其在F1和答案包含率上效果明显。Phi-2模型恢复+6.0 EM、+7.0答案包含率、+0.057 F1；Qwen2.5-1.5B恢复+4.5 EM、+9.0答案包含率、+0.068 F1。通过保留曲线分析竞争累积影响，采用右删失半衰期总结非交叉曲线。结果表明，该协议有效分离出与上下文长度无关的竞争效应，且对不同指标影响程度不同，也受片段长度影响。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems can respond incorrectly even when the correct passage was retrieved. The model must still read the retrieved passages and identify which one contains the answer among others that look relevant. This passage-reading model is called the reader. Does it fail simply because the context is longer or because the other passages genuinely compete with the correct one? We introduce and demonstrate a matched-control protocol for RAG reading: we keep the number and length of passages fixed, but replace hard competitors with less competitive real passages. We apply this control across two compact open models on SQuAD. This replacement partially restores performance, with the strongest effects on F1 and answer inclusion. For Phi-2, this recovers +6.0 EM points, +7.0 answer-inclusion points, and +0.057 F1. For Qwen2.5-1.5B, it recovers +4.5 EM points, +9.0 answer-inclusion points, and +0.068 F1. To track how performance changes as competitors accumulate, we also report retention curves and summarize them with a right-censored half-life when the curves do not cross half-retention. Together, these results show the protocol isolates a competition effect distinct from context length, though the effect is clearer for F1 and answer inclusion than for exact match, and also varies with snippet length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。