用强化学习提升小模型对提示的理解，让小模型也能高效精准重排文档。
ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking
- 通过强化学习优化小模型对任务提示的感知能力。
- 0.5B参数模型在BEIR上超越大型语言模型重排效果。
- 适合资源有限但追求高精度检索的应用场景。
重排序是信息检索和检索增强生成的核心环节，近期大语言模型（LLMs）显著提升了重排序质量。然而，现有方法多依赖大规模模型（>7B参数），计算成本高昂。小语言模型（SLMs）因计算效率高而成为有前景的替代方案。但我们的定量分析显示，SLMs存在表征空间狭窄、表达能力弱，且缺乏微调时难以理解任务提示等问题。为此，我们提出一种两阶段训练方法ProRank，用于基于SLM的文档重排序。通过强化学习提升模型对任务提示的理解，并引入细粒度得分学习以增强表征表达力，进一步提升重排序性能。大量实验表明，ProRank持续优于最先进的开源与专有重排序模型。值得注意的是，我们的0.5B参数ProRank在BEIR基准上超越了强大的大型语言模型重排序器，证明经过适当训练的小模型可在保持计算高效的同时实现卓越的文档重排序表现。
原文摘要 · Abstract (English)
Reranking is fundamental to information retrieval and retrieval-augmented generation, with recent Large Language Models (LLMs) significantly advancing reranking quality. Most current works rely on large-scale LLMs (>7B parameters), presenting high computational costs. Small Language Models (SLMs) offer a promising alternative because of computational efficiency. However, our preliminary quantitative analysis reveals key limitations of SLMs: their representation space is narrow, leading to reduced expressiveness, and they struggle with understanding task prompts without fine-tuning. To address these issues, we introduce a novel two-stage training approach, ProRank, for SLM-based document reranking. We propose using reinforcement learning to improve the understanding of task prompts. Additionally, we introduce fine-grained score learning to enhance representation expressiveness and further improve document reranking quality. Extensive experiments suggest that ProRank consistently outperforms both the most advanced open-source and proprietary reranking models. Notably, our 0.5B ProRank even surpasses powerful LLM reranking models on the BEIR benchmark, establishing that properly trained SLMs can achieve superior document reranking performance while maintaining computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。