不训练的动态检索门控,让大模型只在必要时才查资料。
Retrieval as a Decision: Training-Free Adaptive Gating for Efficient RAG
- 基于大模型生成的草稿前缀,用熵或置信度差判断是否需要检索。
- 在5个问答数据集上减少70%-90%检索次数,延迟降低且准确率不降。
- 无需训练、不增加额外参数,适合资源受限场景使用。
检索增强生成(RAG)虽提升事实准确性,但每次查询都检索会增加计算开销并影响质量。本文提出无需训练的自适应检索门控(TARG),仅通过基础模型生成的短草稿前缀,即可判断是否需检索。TARG利用草稿前缀的前缀逻辑值计算轻量级不确定性得分——包括词元熵、基于前1与前2逻辑值差距的置信度差(单调映射)、或少量随机前缀的小样本方差——当得分超过阈值则触发检索。该机制模型无关,仅增加数十至数百个草稿令牌,无需额外训练或辅助头。在涵盖短答案(NQ-Open、TriviaQA、PopQA)、多跳(MuSiQue)和长文本生成(ASQA)的五个问答基准上,TARG持续推动准确率-效率权衡:相比始终检索(Always-RAG),TARG在保持或提升精确率/召回率的同时,检索频率降低70%-90%,端到端延迟显著下降,且接近从不检索(Never-RAG)的开销。核心发现是,在现代指令微调的大模型中,置信度差信号更鲁棒(熵随模型优化而压缩),小样本方差则提供保守预算优先的替代方案。我们还对门控类型与前缀长度进行了消融实验,并引入Δ-延迟视角明确预算权衡。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) improves factuality but retrieving for every query often hurts quality while inflating tokens and latency. We propose Training-free Adaptive Retrieval Gating (TARG), a single-shot policy that decides when to retrieve using only a short, no-context draft from the base model. From the draft's prefix logits, TARG computes lightweight uncertainty scores-mean token entropy, a margin signal derived from the top-1/top-2 logit gap via a monotone link, or small-N variance across a handful of stochastic prefixes-and triggers retrieval only when the score exceeds a threshold. The gate is model-agnostic, adds only tens to hundreds of draft tokens, and requires no additional training or auxiliary heads. On five QA benchmarks spanning short-answer (NQ-Open, TriviaQA, PopQA), multi-hop (MuSiQue), and long-form (ASQA) tasks, TARG consistently pushes the accuracy-efficiency frontier: compared with Alway-RAG, TARG matches or improves EM/F1 while reducing retrieval by 70-90% and cutting end-to-end latency, and it remains close to Never-RAG in overhead. A central empirical finding is that under modern instruction-tuned LLMs the margin signal is a robust default (entropy compresses as backbones sharpen), with small-N variance offering a conservative, budget-first alternative. We provide ablations over gate type and prefix length and use a $Δ$-latency view to make budget trade-offs explicit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。