用压缩文档前缀提升大模型重排序效率，省时省内存。
DoPR: Reusable Compressed Document Prefixes for Efficient LLM Reranking
- 预先压缩文档为可复用前缀，避免重复计算。
- 在线推理时仅处理查询和评分标记，速度提升8倍以上。
- 适合需要高频重排序的检索系统，如搜索引擎。
大语言模型在文档重排序中表现优异，但针对不同查询重复处理相同文档，造成大量冗余计算。本文提出DoPR框架，将文档表示与查询解耦，提前离线生成并压缩文档前缀状态，后续重排序时直接复用。在线阶段仅需处理查询和评分令牌，文档信息由预存前缀提供。实验在TREC DL、BEIR和BRIGHT数据集上使用Qwen3（0.6B至8B参数）验证，结果显示：文档侧内存减少最高达8.0倍，延迟降低最高达8.04倍，同时保持97.1%–99.5%的平均NDCG@10性能，接近全文档重排序效果。
原文摘要 · Abstract (English)
Large language models (LLMs) are effective rerankers, but pointwise reranking repeatedly processes the same document across different queries, causing substantial redundant document-side computation. We propose \textbf{DoPR}, a compressed document prefix framework that decouples offline document processing from online reranking. DoPR first selects query-independent document representations and converts them into compressed document prefix states, which are precomputed offline and reused whenever the document is retrieved. During online reranking, the model scores each query-document pair by processing only the query and scoring token, with document information supplied by the stored prefix states. This design reduces online cost through both document-side compression and cross-query prefix-state reuse. Experiments on TREC DL, BEIR, and BRIGHT with Qwen3 models from $0.6$B to $8$B show that DoPR achieves up to 8.0$\times$ online document-side memory reduction and up to 8.04$\times$ latency speedup, while retaining \textbf{97.1\%-99.5\%} of the average NDCG@10 of matched full-document rerankers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。