解决长文档RAG系统实时可信验证难题,支持32K token全文校验。
Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systems
- 设计实时验证模块,支持32K token长文档全上下文校验。
- 相比截断验证,全文档校验显著提升未支持回答的检测率。
- 适合需高可信度的企业级搜索与文档助手系统使用。
检索增强生成(RAG)在企业搜索和文档导向助手中的应用日益广泛,要求生成答案严格基于长而复杂的源文档。实践中,验证生成内容是否忠实反映检索文档极为困难:大语言模型虽能处理长上下文,但速度慢、成本高,不适用于交互式服务;轻量级分类器则受限于严格上下文长度,常遗漏截断段落外的证据。本文提出一种集成于生产RAG流水线的实时验证组件,可在延迟约束下实现全文档接地。系统支持最高32K tokens的文档处理,并采用自适应推理策略,在不同负载下平衡响应时间和验证覆盖范围。我们阐述了架构设计、运营权衡及评估方法,表明全上下文验证相比截断验证显著提升未支持回答的检测能力。经验显示,长上下文验证在真实场景中至关重要,基于分块的检查常因文档结构复杂而失效,且延迟预算直接影响模型设计。这些发现为构建可靠的大规模RAG应用提供了实践指导。(模型、基准与代码:https://huggingface.co/llm-semantic-router)
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) is increasingly deployed in enterprise search and document-centric assistants, where responses must be grounded in long and complex source materials. In practice, verifying that generated answers faithfully reflect retrieved documents is difficult: large language models can check long contexts but are too slow and costly for interactive services, while lightweight classifiers operate within strict context limits and frequently miss evidence outside truncated passages. We present the design of a real-time verification component integrated into a production RAG pipeline that enables full-document grounding under latency constraints. The system processes documents up to 32K tokens and employs adaptive inference strategies to balance response time and verification coverage across workloads. We describe the architectural decisions, operational trade-offs, and evaluation methodology used to deploy the verifier, and show that full-context verification substantially improves detection of unsupported responses compared with truncated validation. Our experience highlights when long-context verification is necessary, why chunk-based checking often fails in real documents, and how latency budgets shape model design. These findings provide practical guidance for practitioners building reliable large-scale retrieval-augmented applications. (Model, benchmark, and code: https://huggingface.co/llm-semantic-router)
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。