arXiv:2606.20898cs.IRcs.AI2026-06被引 1

比较两种文档生成方式,发现长文本输入更准但代价高26倍。

The Token Tax of Epistemic Accuracy: Comparing RAG and Long-Context Architectures for Document-Grounded Generative AI Applications

  • 用长上下文加载全文比检索召回更易获得正确答案
  • 长上下文准确率73.1%,高出语义检索65.4%;但令牌成本翻26倍
  • 适合资源充足场景,对预算有限的组织不友好

基于大语言模型的文档增强型助手在高风险、知识密集型工作中日益普及。其有效性可能取决于生成前证据的分配方式。本文比较了两种接地架构:(a) 检索增强生成(RAG),仅检索少量相关段落;(b) 长上下文提示,将整个文档集合放入上下文。我们将二者视为在准确率-成本前沿上的两种“认知访问”模式。使用“认知准确性”衡量模型正确性对正确证据的依赖程度。我们认为更广的访问(通过长上下文)可提升准确性,但需付出“令牌税”(即因输入令牌量大幅增加而导致的成本上升)。通过制造业安全培训的案例研究,采用专家验证基准,评估了972个回答,涵盖三台机器、两个小语言模型及三种检索/上下文提示方法。长上下文提示准确率达73.1%,高于语义RAG的65.4%,但每查询令牌成本高出26倍。我们将其差距解释为更广泛证据访问的令牌税。文章深入讨论了对资源受限组织的启示。

原文摘要 · Abstract (English)

Document-grounded assistants built on large language models are increasingly used in high-stakes, knowledge-intensive work. Their usefulness, however, may depend on how evidence is allocated before generation. We investigate such a claim by comparing two grounding architectures: (a) retrieval-augmented generation (RAG) that retrieves a few relevant passages, and (b) long-context prompting, which loads the whole document collection in context. We view these as two regimes of "epistemic access" on an accuracy--cost frontier. We use "epistemic accuracy" to capture model correctness that depends on having the right evidence. We posit that broader access (via long context) can increase it, but with a "token tax" (i.e., a substantial increase in cost due to larger input token consumption). We probe this framing with a case study in manufacturing safety training. Using an expert-validated benchmark, we evaluate 972 answers across three machines, two small language models, and three retrieval/in-context prompting approaches. Long-context prompting achieved the highest correctness (73.1% vs. 65.4% for semantic RAG), but at 26 times the per-query token cost. We interpret this gap as the token tax of broader evidentiary access. We carefully discuss the implications of our findings for resource-constrained organizations.

RAG长上下文成本分析生成质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。