动态调整检索数量，让大模型问答系统又快又省还合规。
SAGE: SLO-Aware Adaptive Retrieval for Production RAG Systems
- 根据查询难易自动选检索量，避免浪费或不足
- 5秒尾延迟下达标率从30%提至95%，成本降51%
- 一招通用多数据集和模型，不丢准确率
生产环境中的检索增强生成（RAG）系统需满足严格的尾部延迟（P95）和服务水平目标（SLO）及基础设施成本约束。然而，传统检索流水线采用固定检索数量（k），忽略查询难度，导致简单查询过度检索、复杂查询检索不足，迫使运维人员在答案质量与SLO合规性之间权衡。本文提出SAGE，一种基于学习的SLO感知自适应检索策略，可动态为每条查询选择最优检索数量k。SAGE利用初始检索阶段的轻量级特征（如得分分布、排名间隙、词汇信号）进行建模，并通过模仿学习离线训练，学习来自一个近似最优延迟-质量权衡的虚拟专家。推理时无需额外大模型调用，开销极低。在Natural Questions数据集上，当设定5秒P95延迟SLO时，SAGE达到95% SLO合规率，远超最佳静态基线（k=20）的30%；尾延迟降低36%，检索成本减少51%，仅损失2个百分点精确匹配（EM）指标。一个在Natural Questions上训练的单一策略，可泛化至HotpotQA、UnSeenTimeQA及四种大模型家族（Llama、Qwen、Mistral、Gemma），均实现45-52点的SLO提升且无质量下降。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems in production operate under strict service level objectives (SLOs) on tail latency and infrastructure cost. However, standard retrieval pipelines rely on fixed retrieval budgets that ignore query difficulty, over-retrieving for easy queries and under-serving hard ones, forcing operators to trade answer quality against SLO compliance. This paper proposes SAGE, a learned SLO-aware adaptive retrieval policy that dynamically selects the number of passages k per query. SAGE uses lightweight features derived from initial retrieval (e.g., score distributions, rank gaps, lexical signals) and is trained offline via imitation learning from an oracle that approximates optimal latency-quality trade-offs. At inference, it adds no LLM calls and minimal overhead. On Natural Questions, under a 5s P95 latency SLO, SAGE achieves 95% SLO compliance versus 30% for the best static baseline (k=20), reduces P95 latency by 36% and retrieval cost by 51% with only 2 percentage points Exact Match (EM) loss. A single policy trained on Natural Questions generalizes across HotpotQA, UnSeenTimeQA, and four LLM families (Llama, Qwen, Mistral, Gemma), consistently yielding +45-52 point SLO improvements without quality degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。