实测发现,检索融合在真实场景中无法提升问答效果,反而增加延迟。
Scaling Retrieval Augmented Generation with RAG Fusion: Lessons from an Industry Deployment
- 在固定召回深度和重排序预算下测试多种融合方法
- 融合虽提升原始召回率,但重排序后准确率反而下降至0.48
- 适合关注生产系统效率与端到端效果的工程实践者
检索增强生成(RAG)系统常采用多查询检索和倒数排名融合(RRF)等融合技术以提高文档召回率,假设高召回率能带来更好的答案质量。尽管这些方法在孤立的检索基准上表现稳定,但在真实生产环境下的有效性仍不明确。本文评估了在企业知识库上运行的生产级RAG流水线,该流水线受限于固定检索深度、重排序预算和延迟约束。实验表明,虽然融合方法提升了原始召回率,但这些增益在重排序与截断后基本消失。在多个融合配置中,其在知识库级别的Top-k准确率未优于单查询基线,Hit@10从0.51降至0.48。此外,融合因查询重写和更大的候选集引入额外延迟,却未带来下游效果的提升。分析表明,一旦考虑现实中的重排序限制和上下文预算,基于召回的融合技术回报递减。结论是:检索层面的改进并不总能转化为生产RAG系统的端到端收益,应建立兼顾检索质量、系统效率与下游影响的评估框架。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems commonly adopt retrieval fusion techniques such as multi-query retrieval and reciprocal rank fusion (RRF) to increase document recall, under the assumption that higher recall leads to better answer quality. While these methods show consistent gains in isolated retrieval benchmarks, their effectiveness under realistic production constraints remains underexplored. In this work, we evaluate retrieval fusion in a production-style RAG pipeline operating over an enterprise knowledge base, with fixed retrieval depth, re-ranking budgets, and latency constraints. Across multiple fusion configurations, we find that retrieval fusion does increase raw recall, but these gains are largely neutralized after re-ranking and truncation. In our setting, fusion variants fail to outperform single-query baselines on KB-level Top-$k$ accuracy, with Hit@10 decreasing from $0.51$ to $0.48$ in several configurations. Moreover, fusion introduces additional latency overhead due to query rewriting and larger candidate sets, without corresponding improvements in downstream effectiveness. Our analysis suggests that recall-oriented fusion techniques exhibit diminishing returns once realistic re-ranking limits and context budgets are applied. We conclude that retrieval-level improvements do not reliably translate into end-to-end gains in production RAG systems, and argue for evaluation frameworks that jointly consider retrieval quality, system efficiency, and downstream impact.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。