让大模型按需检索，用熵值判断是否查数据库，省资源不丢准确率。
L-RAG: Balancing Context and Retrieval with Entropy-Based Lazy Loading
- 根据模型预测熵值决定是否检索，只在不确定时才查数据库。
- 在SQuAD 2.0上，检索减少26%仍保持76%准确率，单次查询快80-210ms。
- 无需训练，可调阈值平衡速度与精度，适合高并发系统部署。
检索增强生成（RAG）已成为提升大模型事实性输出的主流方法，有效缓解幻觉问题。然而传统RAG采用‘始终检索’策略，无论查询复杂度如何都会查询向量库，带来显著计算开销和延迟，尤其影响高吞吐生产环境。本文提出L-RAG（懒惰检索增强生成），一种基于熵值门控的自适应框架，通过分层上下文管理实现动态检索。L-RAG采用双层架构：先用精简文档摘要处理查询，仅当模型预测熵超过校准阈值（tau）时才触发昂贵的段落检索，表示存在真实不确定性。在SQuAD 2.0（N=500）上使用Phi-2模型实验表明，L-RAG可调节准确率-效率权衡：当tau=0.5时，准确率达78.2%，与标准RAG（77.8%）相当，检索减少8%；当tau=1.0时，检索减少26%，准确率略降至76.0%。延迟分析显示，当检索延迟超过500ms时，每查询节省80-210ms。熵分布分析揭示正确预测（H=1.72）与错误间有统计显著差异（p < 0.001），验证熵作为不确定性信号的可靠性。L-RAG为高效RAG部署提供无需训练的实用方案，使系统架构师可根据需求灵活调节精度与吞吐量。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as the predominant paradigm for grounding Large Language Model outputs in factual knowledge, effectively mitigating hallucinations. However, conventional RAG systems operate under a "retrieve-always" assumption, querying vector databases for every input regardless of query complexity. This static approach incurs substantial computational overhead and inference latency, particularly problematic for high-throughput production deployments. We introduce L-RAG (Lazy Retrieval-Augmented Generation), an adaptive framework that implements hierarchical context management through entropy-based gating. L-RAG employs a two-tier architecture: queries are first processed with a compact document summary, and expensive chunk retrieval is triggered only when the model's predictive entropy exceeds a calibrated threshold, signaling genuine uncertainty. Through experiments on SQuAD 2.0 (N=500) using the Phi-2 model, we demonstrate that L-RAG provides a tunable accuracy-efficiency trade-off: at a conservative threshold (tau=0.5), L-RAG achieves 78.2% accuracy, matching Standard RAG (77.8%), with 8% retrieval reduction; at a balanced threshold (tau=1.0), retrieval reduction increases to 26% with modest accuracy trade-off (76.0%). Latency analysis shows that L-RAG saves 80-210ms per query when retrieval latency exceeds 500ms. Analysis of entropy distributions reveals statistically significant separation (p < 0.001) between correct predictions (H=1.72) and errors (H=2.20), validating entropy as a reliable uncertainty signal. L-RAG offers a practical, training-free approach toward more efficient RAG deployment, providing system architects with a configurable knob to balance accuracy and throughput requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。