提升问答检索准确率，新模型比现有方案高14%。
Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG
- 用大模型重排器优化检索结果排序
- 新模型在问答任务中准确率提升约14%
- 适合关注工业部署效率的开发者
排名模型在提升文本检索系统整体准确性方面起着关键作用。这类多阶段系统通常先使用密集嵌入模型或稀疏词汇索引根据查询检索相关段落，再通过排名模型对候选段落的顺序进行细化，以提高与查询的相关性。本文基准测试了多种公开可用的排名模型，并评估其对排序准确率的影响。研究聚焦于问答任务中的文本检索，这是检索增强生成系统的一种常见应用场景。评估涵盖部分具有工业应用潜力的模型。本文提出一种先进的排名模型 NV-RerankQA-Mistral-4B-v3，相较于其他重排器，其准确率显著提升约14%。同时进行了消融实验，对比不同规模、损失函数和自注意力机制的排名模型微调效果。最后讨论了在实际工业应用中引入排名模型的挑战，尤其是模型大小、排序准确率与索引、服务延迟/吞吐量等系统要求之间的权衡。
原文摘要 · Abstract (English)
Ranking models play a crucial role in enhancing overall accuracy of text retrieval systems. These multi-stage systems typically utilize either dense embedding models or sparse lexical indices to retrieve relevant passages based on a given query, followed by ranking models that refine the ordering of the candidate passages by its relevance to the query. This paper benchmarks various publicly available ranking models and examines their impact on ranking accuracy. We focus on text retrieval for question-answering tasks, a common use case for Retrieval-Augmented Generation systems. Our evaluation benchmarks include models some of which are commercially viable for industrial applications. We introduce a state-of-the-art ranking model, NV-RerankQA-Mistral-4B-v3, which achieves a significant accuracy increase of ~14% compared to pipelines with other rerankers. We also provide an ablation study comparing the fine-tuning of ranking models with different sizes, losses and self-attention mechanisms. Finally, we discuss challenges of text retrieval pipelines with ranking models in real-world industry applications, in particular the trade-offs among model size, ranking accuracy and system requirements like indexing and serving latency / throughput.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。