用推理解释训练小模型，让搜索排序更透明可复现
ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillation
- 用大模型生成文档相关性与比较的推理过程
- 学生模型在MSMARCO和BRIGHT上达到竞品水平
- 适合关注可解释检索与开源模型的研究者
基于查询的相关性重排序是信息检索中的关键任务。传统方法缺乏透明度且依赖专有模型，影响可复现性与可解释性。本文提出Reason-to-Rank（R2R），一种开源重排序方法，通过生成两类推理——直接相关性推理（解释文档如何回应查询）和对比推理（说明一个文档为何比另一个更相关），提升透明度。利用大语言模型（LLMs）作为教师模型生成这些解释，并将知识蒸馏到小型、开放可用的学生模型中。学生模型被训练为生成有意义的推理并进行重排序，在MSMARCO和BRIGHT等多个数据集上取得具有竞争力的性能。实验表明，R2R不仅提升了重排序准确率，还提供了决策过程的深入洞察。通过提供结构化且可解释的解决方案及开放资源，R2R旨在弥合信息检索中效果与透明度之间的差距，促进可复现性与后续研究。
原文摘要 · Abstract (English)
Reranking documents based on their relevance to a given query is a critical task in information retrieval. Traditional reranking methods often lack transparency and rely on proprietary models, hindering reproducibility and interpretability. We propose Reason-to-Rank (R2R), a novel open-source reranking approach that enhances transparency by generating two types of reasoning: direct relevance reasoning, which explains how a document addresses the query, and comparison reasoning, which justifies the relevance of one document over another. We leverage large language models (LLMs) as teacher models to generate these explanations and distill this knowledge into smaller, openly available student models. Our student models are trained to generate meaningful reasoning and rerank documents, achieving competitive performance across multiple datasets, including MSMARCO and BRIGHT. Experiments demonstrate that R2R not only improves reranking accuracy but also provides valuable insights into the decision-making process. By offering a structured and interpretable solution with openly accessible resources, R2R aims to bridge the gap between effectiveness and transparency in information retrieval, fostering reproducibility and further research in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。