一个让LLM做文档重排序的开源框架，开箱即用。
LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Reranking
- 提供统一接口，支持多种LLM和重排方法
- 在多个数据集上验证了不同模型性能，结果可复现
- 适合想快速实验LLM重排的研究者和开发者
利用大语言模型（LLMs）进行文档重排序是近年来热门且有前景的研究方向，众多研究致力于提升其性能与效率。该技术还可广泛应用于搜索引擎或检索增强生成等实际场景。为应对研究与应用的日益增长需求，我们提出一个统一框架 LLM4Ranking，使用户能便捷地使用开源或闭源API型LLM实现不同重排方法。该框架提供简洁可扩展的文档重排序接口，以及易用的评估与微调脚本。我们在多个主流数据集上基于此框架进行了实验，评估了多种模型与方法，提供了可复现的LLM重排序实验结果。代码已公开：https://github.com/liuqi6777/llm4ranking。
原文摘要 · Abstract (English)
Utilizing large language models (LLMs) for document reranking has been a popular and promising research direction in recent years, many studies are dedicated to improving the performance and efficiency of using LLMs for reranking. Besides, it can also be applied in many real-world applications, such as search engines or retrieval-augmented generation. In response to the growing demand for research and application in practice, we introduce a unified framework, \textbf{LLM4Ranking}, which enables users to adopt different ranking methods using open-source or closed-source API-based LLMs. Our framework provides a simple and extensible interface for document reranking with LLMs, as well as easy-to-use evaluation and fine-tuning scripts for this task. We conducted experiments based on this framework and evaluated various models and methods on several widely used datasets, providing reproducibility results on utilizing LLMs for document reranking. Our code is publicly available at https://github.com/liuqi6777/llm4ranking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。