简单检索生成方法在长上下文模型下表现优于复杂多阶段方案。
Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
- 保留原文结构的单阶段检索生成法,避免信息失真
- 在多个长文本问答数据集上超越或持平复杂多阶段方法
- 适合用于评估新RAG系统是否真正带来性能提升
随着可处理数万标记的长上下文语言模型兴起,多阶段检索增强生成(RAG)流水线是否仍比更简单的单阶段方法有明显优势?我们通过控制变量实验,在系统化扩展的令牌预算下,对比了ReadAgent和RAPTOR两种近期多阶段方案与三种基线,包括DOS RAG(保留原始文档结构的RAG),一种保持段落顺序的简单检索-阅读方法。尽管设计简单,DOS RAG在多个长上下文QA基准测试中始终表现匹配或优于更复杂的方案。其优势源于保持源文档真实性和结构、在有效上下文窗口内优先召回关键信息,以及避免额外流水线复杂性。我们建议未来RAG评估应以DOS RAG为简单而强大的基线,搭配先进的嵌入与语言模型,并在相同令牌预算下进行比较,确保新增复杂度由明确性能提升支撑。
原文摘要 · Abstract (English)
With the rise of long-context language models (LMs) capable of processing tens of thousands of tokens in a single context window, do multi-stage retrieval-augmented generation (RAG) pipelines still offer measurable benefits over simpler, single-stage approaches? To assess this question, we conduct a controlled evaluation for QA tasks under systematically scaled token budgets, comparing two recent multi-stage pipelines, ReadAgent and RAPTOR, against three baselines, including DOS RAG (Document's Original Structure RAG), a simple retrieve-then-read method that preserves original passage order. Despite its straightforward design, DOS RAG consistently matches or outperforms more intricate methods on multiple long-context QA benchmarks. We trace this strength to a combination of maintaining source fidelity and document structure, prioritizing recall within effective context windows, and favoring simplicity over added pipeline complexity. We recommend establishing DOS RAG as a simple yet strong baseline for future RAG evaluations, paired with state-of-the-art embedding and language models, and benchmarked under matched token budgets, to ensure that added pipeline complexity is justified by clear performance gains as models continue to improve.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。