用自监督方法训练小模型，让其推理更准、更省参数。
Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking
- 用大模型生成带解释的训练数据，结合强化学习训练小模型。
- 30亿参数模型在BRIGHT上排名第三，优于20倍大的模型。
- 推理时生成解释，提升小模型的推理能力，适合资源有限场景。
我们提出一种新方法，训练小型语言模型进行推理密集型文档重排。现有方法多依赖昂贵的人工标注或大型黑箱模型，而本方法利用网络数据和教师大模型，自动构建高质量带相关性解释的训练样本。将文档排序建模为强化学习问题，激励显式推理能力，训练出一个30亿参数的小型语言模型，在BRIGHT基准测试中达到领先性能。该模型在排行榜上位列第三，参数量远低于其他方法，且表现优于参数超过20倍大的模型。大量实验表明，推理时生成解释而非直接预测相关性分数，能更有效提升小模型的推理能力。该方法具有自监督特性，为现代信息检索系统提供可扩展、可解释的解决方案。
原文摘要 · Abstract (English)
We present a novel approach for training small language models for reasoning-intensive document ranking that combines knowledge distillation with reinforcement learning optimization. While existing methods often rely on expensive human annotations or large black-box language models, our methodology leverages web data and a teacher LLM to automatically generate high-quality training examples with relevance explanations. By framing document ranking as a reinforcement learning problem and incentivizing explicit reasoning capabilities, we train a compact 3B parameter language model that achieves state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on the leaderboard while using substantially fewer parameters than other approaches, outperforming models that are over 20 times larger. Through extensive experiments, we demonstrate that generating explanations during inference, rather than directly predicting relevance scores, enables more effective reasoning with smaller language models. The self-supervised nature of our method offers a scalable and interpretable solution for modern information retrieval systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。