通过挖掘难负例提升跨编码器重排模型性能,增强企业级检索系统效果。
Enhancing Retrieval Performance: An Ensemble Approach For Hard Negative Mining
- 设计高效难负例挖掘方法,结合领域数据优化训练过程。
- 相比随机采样,使用难负例使检索准确率显著提升。
- 适用于RAG和推理代理等先进大模型系统的检索模块优化。
排序在信息检索研究中始终是核心关注点。检索与排序模型是网络搜索、开放域问答、企业域问答及基于文本的推荐系统的基础。通常,这些模型在包含正负片段的三元组上进行训练,其中正负样本由二元相关性标注。然而,在重新排序大量潜在相关文档时,需要更精细的相关性理解。虽然通过用户行为(如点击)收集正例较为简单,但从数百万甚至数十亿文档中识别合适的负例仍具挑战。生成大量负例对维持模型质量至关重要。现有文献提出多种从大规模语料库中选择合适负例的方法。本文聚焦于硬负例在跨编码器模型训练中的关键作用,解释了为何硬负例采样相比随机采样能带来性能提升。我们提出一种稳健的难负例挖掘技术,用于在企业数据集上高效训练跨编码器重排模型,该模型具有领域特定上下文。本研究提供了一种新视角,通过同时学习相似性和差异性,提升检索模型性能,进而影响RAG和ReAct等先进大模型系统的表现。
原文摘要 · Abstract (English)
Ranking consistently emerges as a primary focus in information retrieval research. Retrieval and ranking models serve as the foundation for numerous applications, including web search, open domain QA, enterprise domain QA, and text-based recommender systems. Typically, these models undergo training on triplets consisting of binary relevance assignments, comprising one positive and one negative passage. However, their utilization involves a context where a significantly more nuanced understanding of relevance is necessary, especially when re-ranking a large pool of potentially relevant passages. Although collecting positive examples through user feedback like impressions or clicks is straightforward, identifying suitable negative pairs from a vast pool of possibly millions or even billions of documents possess a greater challenge. Generating a substantial number of negative pairs is often necessary to maintain the high quality of the model. Several approaches have been suggested in literature to tackle the issue of selecting suitable negative pairs from an extensive corpus. This study focuses on explaining the crucial role of hard negatives in the training process of cross-encoder models, specifically aiming to explain the performance gains observed with hard negative sampling compared to random sampling. We have developed a robust hard negative mining technique for efficient training of cross-encoder re-rank models on an enterprise dataset which has domain specific context. We provide a novel perspective to enhance retrieval models, ultimately influencing the performance of advanced LLM systems like Retrieval-Augmented Generation (RAG) and Reasoning and Action Agents (ReAct). The proposed approach demonstrates that learning both similarity and dissimilarity simultaneously with cross-encoders improves performance of retrieval systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。