用极轻量查询编码实现千倍加速的LLM文本检索系统
LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference
- 查询编码仅需嵌入查找，无需运行完整LLM
- 查询编码速度提升超1000倍，端到端吞吐增10倍
- 在多种任务上保持95%以上检索性能，适合高并发场景
基于大语言模型（LLMs）的文本检索通过向量相似性匹配相关文档。文档离线预编码，而查询实时到达，对在线查询编码器的效率要求极高。尽管LLMs显著提升了检索能力，但部署参数量大的完整模型会严重降低查询推理吞吐并增加资源消耗。本文提出LightRetriever，一种新型轻量级查询编码的LLM检索架构：保留全尺寸LLM用于文档编码，将查询编码工作量降至不超过一次嵌入查找。与在A800 GPU上部署完整LLM相比，该方法使查询编码速度提升超过1000倍，端到端检索吞吐提高超过10倍。在大规模检索基准上的实验表明，LightRetriever在多种任务中均表现出良好泛化能力，平均保持95%的检索性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs)-based text retrieval retrieves documents relevant to search queries based on vector similarities. Documents are pre-encoded offline, while queries arrive in real-time, necessitating an efficient online query encoder. Although LLMs significantly enhance retrieval capabilities, serving deeply parameterized LLMs slows down query inference throughput and increases demands for online deployment resources. In this paper, we propose LightRetriever, a novel LLM-based retriever with extremely lightweight query encoders. Our method retains a full-sized LLM for document encoding, but reduces the workload of query encoding to no more than an embedding lookup. Compared to serving a full LLM on an A800 GPU, our method achieves over 1000x speedup in query encoding and over 10x increase in end-to-end retrieval throughput. Extensive experiments on large-scale retrieval benchmarks show that LightRetriever generalizes well across diverse tasks, maintaining an average of 95% retrieval performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。