根据查询复杂度动态选择检索深度,降低大模型成本与延迟。
Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs

- 按查询类型自动匹配不同检索深度和生成配置的策略组合
- 节省26%计费令牌,降低34%平均延迟,答案质量不变
- 适合需要控制成本与响应速度的生产级大模型应用
检索增强生成(RAG)面临深层检索提升事实准确性但增加令牌开销和延迟的根本矛盾。静态配置无法适应不同查询负载——简单定义类问题浪费资源,复杂分析类问题又因检索过浅而不足。本文提出成本感知RAG(CA-RAG),一个基于每查询路由的框架,从离散策略包中选择最优组合(检索深度从无检索到前k=10稠密检索,搭配固定生成配置),通过最大化包含质量预估与归一化延迟及计费令牌惩罚的综合效用函数实现。系统基于FAISS稠密检索与OpenAI API构建,在28个查询的基准上验证,动态启用全部策略包,相比始终使用重型检索减少26%计费令牌,相比始终直接推理降低34%平均延迟,同时保持等效答案质量。每查询差异分析显示节省集中在简单查询,提示应引入复杂度感知约束。敏感性分析表明,同一策略包目录可通过权重调整支持多种成本-延迟-质量权衡点。所有结果均来自日志CSV文件,保证可复现性。CA-RAG为成本敏感的大模型部署提供透明、可审计基础。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) faces a fundamental three-way tension: deeper retrieval improves factual grounding but inflates token costs and end-to-end latency. Static retrieval configurations cannot resolve this tension across heterogeneous query workloads -- simple definitional queries waste budget on unnecessary context, while complex analytical prompts are underserved by shallow retrieval. This paper introduces \emph{Cost-Aware RAG} (CA-RAG), a per-query routing framework that selects from a discrete catalog of \emph{strategy bundles} -- each coupling a retrieval depth (from retrieval-free direct inference to top-$k{=}10$ dense retrieval) with a fixed generation profile -- by maximizing a scalar utility that linearly combines an estimated quality prior with normalized penalties for predicted latency and total billed tokens. CA-RAG is implemented with FAISS-backed dense retrieval and OpenAI chat/embedding APIs, and evaluated on a 28-query benchmark spanning four bundles. The router dynamically exercises all bundles, achieving \textbf{26\% fewer billed tokens} than always-heavy retrieval and \textbf{34\% lower mean latency} than always-direct inference while maintaining equivalent answer quality. Per-query delta analysis reveals that savings are non-uniform and concentrated in simpler queries, motivating complexity-aware guardrails. Sensitivity analysis confirms that the same bundle catalog supports multiple cost-latency-quality operating points through weight adjustment alone. All results are generated directly from logged CSV artifacts for full reproducibility. CA-RAG provides a transparent, auditable foundation for cost-conscious LLM deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。