用微调的小模型自动标注相关性，提升搜索排序效果。
Augmented Relevance Datasets with Fine-Tuned Small LLMs
- 用小规模LLM微调后自动判断查询与文档相关性
- 相比某些闭源模型表现更好，显著提升排序模型性能
- 适合需要高效构建高质量数据集的搜索系统优化场景
构建高质量数据集和标注查询-文档相关性是关键但资源密集的任务，需详细指南和大量人工投入。本文探索使用微调的小型大语言模型（LLMs）自动化相关性评估，重点通过增强训练数据集来提升排序模型性能。我们对小型LLM进行微调以改进相关性判断，从而提高下游排序模型训练的数据质量。实验表明，这些微调后的小型LLM不仅在我们的数据集上超越某些闭源模型，还显著提升了排序模型的表现。结果凸显了利用小型LLM实现高效、可扩展的数据集增强的潜力，为搜索引擎优化提供了实用解决方案。
原文摘要 · Abstract (English)
Building high-quality datasets and labeling query-document relevance are essential yet resource-intensive tasks, requiring detailed guidelines and substantial effort from human annotators. This paper explores the use of small, fine-tuned large language models (LLMs) to automate relevance assessment, with a focus on improving ranking models' performance by augmenting their training dataset. We fine-tuned small LLMs to enhance relevance assessments, thereby improving dataset creation quality for downstream ranking model training. Our experiments demonstrate that these fine-tuned small LLMs not only outperform certain closed source models on our dataset but also lead to substantial improvements in ranking model performance. These results highlight the potential of leveraging small LLMs for efficient and scalable dataset augmentation, providing a practical solution for search engine optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。