用压缩嵌入重排文档，让大模型更省力高效
Reranking with Compressed Document Representation
- 将文档压缩为固定大小嵌入，减少输入量
- 蒸馏训练使小模型在长文档上超越大模型
- 适合需要高效处理长文本的检索场景
重排是优化第一阶段检索器输出的重要步骤，但常因计算开销大而受限，尤其是使用大语言模型时。受RAG中文档压缩技术启发，我们通过将文档压缩为固定大小的嵌入表示来减小输入规模，并利用知识蒸馏训练重排器使用压缩后的输入。尽管基础模型规模达十亿级，但经过训练的重排器在使用压缩输入时，其效果与效率均能与更小的重排器竞争，尤其在处理长文档时表现突出。考虑到文本压缩器仍处于发展初期，该方法展现出巨大潜力。
原文摘要 · Abstract (English)
Reranking, the process of refining the output of a first-stage retriever, is often considered computationally expensive, especially with Large Language Models. Borrowing from recent advances in document compression for RAG, we reduce the input size by compressing documents into fixed-size embedding representations. We then teach a reranker to use compressed inputs by distillation. Although based on a billion-size model, our trained reranker using this compressed input can challenge smaller rerankers in terms of both effectiveness and efficiency, especially for long documents. Given that text compressors are still in their early development stages, we view this approach as promising.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。