开源工具包，让大模型轻松实现文档重排序。
RankLLM: A Python Package for Reranking with LLMs
- 模块化设计，支持多种大模型进行重排序。
- 可复现RankGPT等多篇论文结果，提升研究效率。
- 适合做RAG系统开发或检索优化的研究人员。
大型语言模型（LLM）作为多阶段检索系统中的重排序器，在学术界和工业界日益流行。这类模型通过精心设计的提示词对检索到的文档候选列表进行精炼，常用于检索增强生成（RAG）应用中。本文介绍RankLLM，一个开源的Python工具包，支持自定义重排序工作流，兼容专有与开源大模型，具备高度可配置性。为提升易用性，RankLLM提供与Pyserini的可选集成，并内置多阶段流水线评估功能。此外，其分析模块可深入检查输入提示与大模型输出，缓解大模型API的可靠性问题及混合专家（MoE）模型的非确定性行为。本文详述了RankLLM架构，附带分步指南与示例代码，并复现了RankGPT、LRL、RankVicuna、RankZephyr等近期模型的结果。该工具包支持常见推理框架与广泛的大模型，便于快速复现报告结果，加速科研与实际应用。完整仓库地址：rankllm.ai，可通过PyPI安装。
原文摘要 · Abstract (English)
The adoption of large language models (LLMs) as rerankers in multi-stage retrieval systems has gained significant traction in academia and industry. These models refine a candidate list of retrieved documents, often through carefully designed prompts, and are typically used in applications built on retrieval-augmented generation (RAG). This paper introduces RankLLM, an open-source Python package for reranking that is modular, highly configurable, and supports both proprietary and open-source LLMs in customized reranking workflows. To improve usability, RankLLM features optional integration with Pyserini for retrieval and provides integrated evaluation for multi-stage pipelines. Additionally, RankLLM includes a module for detailed analysis of input prompts and LLM responses, addressing reliability concerns with LLM APIs and non-deterministic behavior in Mixture-of-Experts (MoE) models. This paper presents the architecture of RankLLM, along with a detailed step-by-step guide and sample code. We reproduce results from RankGPT, LRL, RankVicuna, RankZephyr, and other recent models. RankLLM integrates with common inference frameworks and a wide range of LLMs. This compatibility allows for quick reproduction of reported results, helping to speed up both research and real-world applications. The complete repository is available at rankllm.ai, and the package can be installed via PyPI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。