用少量高质量数据训练大模型，实现高效推理型信息重排序
LimRank: Less is More for Reasoning-Intensive Information Reranking
- 仅需5%的标注数据，通过合成数据提升模型性能
- 在BRIGHT和FollowIR两个挑战性任务上表现媲美主流方法
- 适合需要低资源、强泛化能力的检索与问答场景
现有方法通常依赖大规模微调来适配大模型进行信息重排序，计算成本高昂。本文证明，现代大模型仅需少量高质量监督即可有效适配。为此，我们设计了LIMRANK-SYNTHESIZER——一个可复用、开源的合成数据生成管道，用于生成多样、具有挑战性且真实的重排序样本。利用该合成数据，我们微调了重排序模型LIMRANK。在两个挑战性基准测试BRIGHT（推理密集型检索）和FollowIR（指令遵循型检索）上评估表明，LIMRANK仅使用先前工作1/20的数据量，仍能达到竞争力的表现。进一步消融实验验证了LIMRANK-SYNTHESIZER的有效性及LIMRANK在下游任务中的强大泛化能力，包括科学文献搜索与知识密集型问题求解的检索增强生成。
原文摘要 · Abstract (English)
Existing approaches typically rely on large-scale fine-tuning to adapt LLMs for information reranking tasks, which is computationally expensive. In this work, we demonstrate that modern LLMs can be effectively adapted using only minimal, high-quality supervision. To enable this, we design LIMRANK-SYNTHESIZER, a reusable and open-source pipeline for generating diverse, challenging, and realistic reranking examples. Using this synthetic data, we fine-tune our reranker model, LIMRANK. We evaluate LIMRANK on two challenging benchmarks, i.e., BRIGHT for reasoning-intensive retrieval and FollowIR for instruction-following retrieval. Our experiments demonstrate that LIMRANK achieves competitive performance, while being trained on less than 5% of the data typically used in prior work. Further ablation studies demonstrate the effectiveness of LIMRANK-SYNTHESIZER and the strong generalization capabilities of LIMRANK across downstream tasks, including scientific literature search and retrieval-augmented generation for knowledge-intensive problem solving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。