发现检索增强系统中存在覆盖幻觉,提出后检索级联策略优化性能。
The Coverage Illusion: From Pre-retrieval Routing Failure to Post-retrieval Cascades in a Production RAG System

- 基于真实流量评估五种检索流程,发现合成查询与实际查询需求差异巨大。
- 提出后检索级联机制,72.2%用户无需大模型增广,延迟降31.8%,效果提升0.140分。
- 无需训练或额外服务,适合生产环境部署,解决预检索路由失效问题。
在现代检索增强生成(RAG)流水线中,诸如HyDE和查询扩展等查询增强方法被应用于所有查询,导致显著的LLM推理开销和端到端延迟。然而,这种开销在真实生产流量中的实证依据仍不充分。我们以丹麦国家百科全书为案例,评估了五种检索工作流在20,000个真实与合成查询-工作流对上的表现。合成查询表明,超过90%的查询需使用大模型增强才能实现高检索覆盖率;但在生产环境的延迟策略下,仅27.8%的真实用户查询需要此类增强。这一差距被称为‘覆盖幻觉’,源于合成与真实查询分布的结构性错配。预检索路由无法解决该问题,因为是否需要大模型增强只能在索引搜索后才可确定。我们评估四种机器学习范式后证实此结论。由于该覆盖差距仅凭查询本身无法检测,因此我们提出一种后检索级联策略:按成本从低到高运行工作流,仅当某步无文档返回时才升级至大模型增强。该策略无需训练开销或额外服务基础设施,在保持质量的前提下,相比始终使用HyDE方案,复合总体得分提升+0.140分,延迟降低31.8%,且72.2%的真实用户查询无需大模型处理。
原文摘要 · Abstract (English)
In modern RAG pipelines, query augmentation methods such as HyDE and query expansion are applied to every query, resulting in substantial LLM inference costs and increased end-to-end latency. The empirical justification for this overhead in real production traffic remains largely unexplored. We present a case study of the Danish National Encyclopedia, evaluating five retrieval workflows over 20,000 query-workflow pairs from production traffic and synthetic conditions. In this system, synthetic queries suggest that LLM augmentation is needed for over 90% of queries to achieve high retrieval coverage. However, under our production deferral policy, only 27.8% of real user queries need LLM augmentation. We call this gap the Coverage Illusion and attribute it to a structural mismatch between synthetic and real query distributions. Pre-retrieval routing cannot resolve this gap, as the need for LLM augmentation is only revealed after searching the index, a result confirmed by our evaluation of four machine learning paradigms. The coverage gap, undetectable from the query alone, motivates a post-retrieval cascade that runs workflows in cheapest-first order and escalates to LLM augmentation only when a step returns no documents. Operating entirely without training overhead or secondary serving infrastructure, the cascade improves quality by +0.140 Composite Overall points over Always-HyDE, reduces latency by 31.8%, and serves 72.2% of real user queries without LLM augmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。