用单次扫描处理超长文档，高效保持上下文连贯性。
Single-Pass Document Scanning for Question Answering
- 单遍线性扫描全文，避免分块导致的上下文断裂
- 在41个基准上超越分块嵌入方法，计算成本仅为大模型的几分之一
- 适合需要长文本理解的问答系统，尤其适用于海量文档
处理超长文档进行问答任务极具挑战：基于分块的嵌入方法常丢失重要全局上下文，而全上下文变换器对数十万词元的文本计算开销巨大。本文提出一种单次扫描文档的方法，以线性时间处理全文，在保持全局连贯性的同时判断句子与查询的相关性。在41个问答基准上，该方法始终优于分块嵌入方法，并以极低计算成本媲美大型语言模型。通过依赖完整前序上下文而不分块，该方法有效保留了长文档中的全局语义连贯性。整体而言，单次扫描文档法为大规模文本问答提供了一种简洁高效的解决方案。所有代码、数据集和模型权重均开源于https://github.com/MambaRetriever/MambaRetriever。
原文摘要 · Abstract (English)
Handling extremely large documents for question answering is challenging: chunk-based embedding methods often lose track of important global context, while full-context transformers can be prohibitively expensive for hundreds of thousands of tokens. We propose a single-pass document scanning approach that processes the entire text in linear time, preserving global coherence while deciding which sentences are most relevant to the query. On 41 QA benchmarks, our single-pass scanner consistently outperforms chunk-based embedding methods and competes with large language models at a fraction of the computational cost. By conditioning on the entire preceding context without chunk breaks, the method preserves global coherence, which is especially important for long documents. Overall, single-pass document scanning offers a simple solution for question answering over massive text. All code, datasets, and model checkpoints are available at https://github.com/MambaRetriever/MambaRetriever
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。