用置信度路由机制，让85%的查询快速回答，效率提升120倍。
EverydayGPT: Confidence-Gated Routing for Efficient and Safe Hybrid GPT-RAG Conversational QA
- 根据检索距离和提取充分性，动态决定是否调用大模型
- 85%查询无需大模型，平均延迟降低6.3倍，最高降120倍
- 适合资源有限场景，兼顾效率与答案可信
标准RAG系统对每个查询无差别地执行检索与生成，导致计算浪费并引入低质量上下文。本文提出EverydayGPT，基于置信度门控路由（CGR）机制，将路由决策建模为检索距离与提取充分性的联合策略。系统采用从头训练的205M参数GPT，基于100亿个FineWeb-Edu token。CGR使85%的查询通过快速RAG提取解决（约45毫秒），避免调用耗时的GPT路径（约5.9秒），在多数查询上实现超120倍的延迟降低，同时保持答案质量。在500题领域内基准测试中，系统F1达0.226 ± 0.004，优于GPT-only的0.171和无条件RAG的0.210。相较于强基线，性能提升虽温和但稳定，效率提升显著。结构化溯源审计显示采样集无未支持声明，且明确标注范围限制。本工作聚焦资源约束下的路由策略研究，不宣称达到顶尖性能。
原文摘要 · Abstract (English)
Standard Retrieval-Augmented Generation (RAG) pipelines route every query through retrieval and generation unconditionally, incurring unnecessary computation and propagating low-quality context to the generator. We introduce EverydayGPT, a lightweight conversational QA system built around a Confidence-Gated Routing (CGR) mechanism that formalises the routing decision as a joint policy over retrieval distance and extraction adequacy. The backbone is a 205M-parameter GPT trained from scratch on 10B tokens of FineWeb-Edu. CGR avoids invoking the costly GPT pathway (~5.9s) for 85 percent of queries by resolving them via fast RAG extraction (~45 ms), yielding over 120x latency reduction on the majority of queries while maintaining answer quality. On a 500-question in-domain benchmark, the system achieves F1 = 0.226 +/- 0.004 compared to 0.171 for GPT-only and 0.210 for unconditional RAG. Gains over strong baselines are modest but consistent, while efficiency improvements are substantial (6.3x mean latency reduction). A structured grounding audit finds no unsupported claims in the sampled set, with explicit scope limitations. We position this work as a study of routing strategies under resource constraints rather than a claim of state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。