用大模型生成数据和标签，让小模型高效提升搜索排序效果
Enhancing Transformer-Based Rerankers with Synthetic Data and LLM-Based Supervision
- 用大模型从专业语料生成合成查询，自动标注正负样本
- 在MedQuAD上比基线提升显著，跨领域泛化能力好
- 避免人工标注，计算成本低，适合实际部署
有效的文档重排序对提升各类应用中的搜索相关性至关重要。尽管大语言模型（LLMs）凭借其深层语义理解与推理能力在重排序任务中表现优异，但其高昂的计算开销使其难以在多数实际场景中部署。微调更小、专用的模型是更高效的替代方案，但通常依赖稀缺的人工标注数据。为此，我们提出一种新流程，无需人工标注的查询-文档对。该方法利用大模型从领域特定语料生成合成查询，并使用基于大模型的分类器标注正例与难负例。由此构建的合成数据集用于微调小型Transformer模型，采用局部对比估计（LCE）损失进行对比学习。在MedQuAD数据集上的实验表明，该方法显著提升了域内性能，并在域外任务上具有良好泛化能力。通过将大模型用于数据生成与监督而非推理，我们在保持强重排序能力的同时大幅降低计算成本。
原文摘要 · Abstract (English)
Effective document reranking is essential for improving search relevance across diverse applications. While Large Language Models (LLMs) excel at reranking due to their deep semantic understanding and reasoning, their high computational cost makes them impractical for many real-world deployments. Fine-tuning smaller, task-specific models is a more efficient alternative but typically depends on scarce, manually labeled data. To overcome this, we propose a novel pipeline that eliminates the need for human-labeled query-document pairs. Our method uses LLMs to generate synthetic queries from domain-specific corpora and employs an LLM-based classifier to label positive and hard-negative pairs. This synthetic dataset is then used to fine-tune a smaller transformer model with contrastive learning using Localized Contrastive Estimation (LCE) loss. Experiments on the MedQuAD dataset show that our approach significantly boosts in-domain performance and generalizes well to out-of-domain tasks. By using LLMs for data generation and supervision rather than inference, we reduce computational costs while maintaining strong reranking capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。