arXiv:2606.29959cs.IRcs.CL2026-06

根据置信度动态分配检索预算,提升问答系统效率与准确性。

Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation

论文配图:Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation
图 1 · 摘自论文原文
  • 用校准概率决定是否检索、检索多少内容或直接回答。
  • 在TriviaQA等数据集上,错误率下降超70%,校准后概率更可信。
  • 适合追求高效推理、关注延迟与资源平衡的部署场景。

检索增强生成(RAG)通常对每个查询固定检索若干段落,但当答案已知时这会浪费资源,无关内容还可能干扰模型。本文提出将自适应RAG建模为校准后的检索预算分配:根据查询判断是否闭门作答、仅检索1段、检索5段或放弃回答。核心贡献是将序列对数概率和前缀对数几率信号转化为正确性概率,用于分级上下文选择、选择性回避及显式权衡延迟与令牌开销。在TriviaQA、Natural Questions和MS MARCO等核心问答任务上,交叉验证校准使序列对数概率的ECE从0.275降至0.062(TriviaQA),0.643降至0.009(NQ),0.711降至0.031(MS MARCO)。分级检索显著提升完整上下文与段落预算边界表现,而检索调用AUC与二元门控基本持平(因k=1仍需一次检索)。保留训练/验证/测试阈值实验确定可部署操作点。在精度匹配的前沿点上,实测成本模型显示:门控并非普遍更快——在Qwen3-8B上延迟增加约27%,但在Qwen3-32B上节省约8%。结果表明,校准置信度应视为在任务与系统约束下分配检索预算的通用接口。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) typically retrieves a fixed number of passages for every query. This is wasteful when the reader already knows the answer, and it can be harmful when irrelevant or partially relevant passages distract the reader. We formulate adaptive RAG as calibrated retrieval-budget allocation: given a query, decide whether to answer closed-book, retrieve a compact context (k=1), retrieve a full context (k=5), or abstain. The contribution is a probability interface rather than a new raw uncertainty signal. We calibrate sequence log-probability and prefix-logit uncertainty signals into probabilities of correctness, then use these probabilities for graded context selection, selective abstention, and explicit latency/token trade-offs. Across core QA experiments on TriviaQA, Natural Questions, and MS MARCO, with auxiliary PopQA motivation and Qwen/Llama family checks, diagnostic out-of-fold calibration improves probability quality dramatically: for sequence log-probability, ECE drops from 0.275 to 0.062 on TriviaQA, 0.643 to 0.009 on NQ, and 0.711 to 0.031 on MS MARCO. Graded retrieval improves full-context and passage-budget frontiers for both our signal and TARG-style prefix entropy/margin, while retrieval-call AUC remains essentially tied with binary gating because k=1 is still a retrieval call. Held-out train/validation/test threshold experiments report deployable operating points. At matched-accuracy frontier operating points, a measured cost model reveals that gating is not universally faster: it increases latency by about 27% on Qwen3-8B but saves about 8% on Qwen3-32B. These results support a nuanced view of adaptive RAG: calibrated confidence is best understood as a reusable interface for allocating retrieval budget under task and system constraints.

RAG检索优化置信度校准高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。