用压缩文档嵌入实现高效列表重排序，速度提升3-18倍。
Efficient Listwise Reranking with Compressed Document Representations
- 将文档压缩为固定长度的多标记嵌入，结合列表重排序机制。
- 80亿参数模型速度比0.6-40亿参数模型快3至18倍，效果相当或更优。
- 特别适合长文档重排序任务，效率优势更明显。
重排序是优化第一阶段检索器输出的重要步骤，但通常计算成本较高，尤其是使用大语言模型（LLM）时。为缓解这一问题，常用方法包括使用较小的LLM或控制输入长度。受检索增强生成（RAG）中文档压缩技术的启发，我们提出RRK——一种高效且有效的列表重排序方法，将文档压缩为多标记固定大小的嵌入表示。通过简单的蒸馏训练，该方法结合丰富的压缩表示与列表重排序，实现了高效率与高有效性。特别是，我们的80亿参数模型运行速度比0.6-40亿参数的小型重排序器快3至18倍，同时在效果上达到或超过它们。在长文档基准测试中，其效率优势更加显著。
原文摘要 · Abstract (English)
Reranking, the process of refining the output from a first-stage retriever, is often considered computationally expensive, especially when using Large Language Models (LLMs). A common approach to mitigate this cost involves utilizing smaller LLMs or controlling input length. Inspired by recent advances in document compression for retrieval-augmented generation (RAG), we introduce RRK, an efficient and effective listwise reranker compressing documents into multi-token fixed-size embedding representations. Our simple training via distillation shows that this combination of rich compressed representations and listwise reranking yields a highly efficient and effective system. In particular, our 8B-parameter model runs 3x-18x faster than smaller rerankers (0.6-4B parameters) while matching or outperforming them in effectiveness. The efficiency gains are even more striking on long-document benchmarks, where RRK widens its advantage further.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。