轻量级框架让Transformer模型轻松用于信息检索，省去复杂工程负担。
Lightning IR: Straightforward Fine-tuning and Inference of Transformer-based Language Models for Information Retrieval
- 基于PyTorch Lightning构建模块化架构，支持检索全流程
- 开箱即用，可直接用于微调、索引、搜索与重排序
- 开源可复现，适合研究者快速搭建检索系统
大量基于Transformer的语言模型已被提出用于信息检索任务。然而，将这些模型集成到检索流水线中通常复杂且需要大量工程工作。本文提出Lightning IR，一个基于PyTorch Lightning的易用框架，适用于在检索场景中应用Transformer语言模型。该框架提供模块化、可扩展的架构，支持检索流水线的所有阶段：从微调、索引到搜索和重排序。设计上注重可扩展性和可复现性，已作为开源项目发布：https://github.com/webis-de/lightning-ir。
原文摘要 · Abstract (English)
A wide range of transformer-based language models have been proposed for information retrieval tasks. However, including transformer-based models in retrieval pipelines is often complex and requires substantial engineering effort. In this paper, we introduce Lightning IR, an easy-to-use PyTorch Lightning-based framework for applying transformer-based language models in retrieval scenarios. Lightning IR provides a modular and extensible architecture that supports all stages of a retrieval pipeline: from fine-tuning and indexing to searching and re-ranking. Designed to be scalable and reproducible, Lightning IR is available as open-source: https://github.com/webis-de/lightning-ir.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。