提出可识别干扰的检索方法，让大模型更高效地用知识
Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding
- 设计能感知干扰的检索机制，动态筛选有用信息
- 在六项测试中优于长上下文方法，节省大量token
- 适合资源有限但需精准知识的大模型应用
检索增强生成（RAG）是将大语言模型（LLMs）与外部实时信息结合的框架。然而，随着上下文窗口扩大至128K token以上，直接提供全文上下文成为替代方案。但该方法存在显著缺陷：（i）处理大型冗余文本时效率低下；（ii）加剧‘中间遗忘’现象；（iii）在模型容量受限时放大干扰，降低输出质量。本文提出LDAR（学习干扰感知检索），一种自适应检索器，通过识别并规避干扰段落，实现更高性能与更低的token消耗。在多种LLM架构及六个知识密集型基准上的实验证明，该方法有效且鲁棒，强调了信息覆盖与干扰控制之间的平衡至关重要。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is a framework for grounding Large Language Models (LLMs) in external, up-to-date information. However, recent advancements in context window size allow LLMs to process inputs of up to 128K tokens or more, offering an alternative strategy: supplying the full document context directly to the model, rather than relying on RAG to retrieve a subset of contexts. Nevertheless, this emerging alternative strategy has notable limitations: (i) it is token-inefficient to handle large and potentially redundant contexts; (ii) it exacerbates the `lost in the middle' phenomenon; and (iii) under limited model capacity, it amplifies distraction, ultimately degrading LLM output quality. In this paper, we propose LDAR (Learning Distraction-Aware Retrieval), an adaptive retriever that learns to retrieve contexts in a way that mitigates interference from distracting passages, thereby achieving significantly higher performance with reduced token usage compared to long-context approaches. Extensive experiments across diverse LLM architectures and six knowledge-intensive benchmarks demonstrate the effectiveness and robustness of our approach, highlighting the importance of balancing the trade-off between information coverage and distraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。