arXiv:2412.17250cs.IR2024-12被引 21

用大模型生成高质量难负样本,提升稠密检索效果

SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval

  • 用多属性自省提示词引导大模型生成难负样本
  • 混合合成负样本与传统负样本,提升训练稳定性
  • 在5个数据集上验证有效,适合检索系统优化者

稠密检索性能受负样本质量显著影响。传统方法依赖简单负样本采样或通过外部检索器挖掘难负样本,但前者难以捕捉正负样本边界,后者易引入误负样本,导致性能下降和训练不稳定。本文提出利用大语言模型生成语义丰富且多样化的高质量难负样本。首先设计多属性自省提示策略,指导大模型生成难负样本;再采用混合采样策略,将合成负样本与传统负样本结合,稳定训练过程并提升检索性能。在五个基准数据集上的实验表明该方法有效,代码已公开。

原文摘要 · Abstract (English)

The performance of Dense retrieval (DR) is significantly influenced by the quality of negative sampling. Traditional DR methods primarily depend on naive negative sampling techniques or on mining hard negatives through external retriever and meticulously crafted strategies. However, naive negative sampling often fails to adequately capture the accurate boundaries between positive and negative samples, whereas existing hard negative sampling methods are prone to false negatives, resulting in performance degradation and training instability. Recent advancements in large language models (LLMs) offer an innovative solution to these challenges by generating contextually rich and diverse negative samples. In this work, we present a framework that harnesses LLMs to synthesize high-quality hard negative samples. We first devise a \textit{multi-attribute self-reflection prompting strategy} to direct LLMs in hard negative sample generation. Then, we implement a \textit{hybrid sampling strategy} that integrates these synthetic negatives with traditionally retrieved negatives, thereby stabilizing the training process and improving retrieval performance. Extensive experiments on five benchmark datasets demonstrate the efficacy of our approach, and code is also publicly available.

稠密检索大模型负样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。