用大模型生成假负例训练检索模型，效果却不如传统方法。
Don't Retrieve, Generate: Prompting LLMs for Synthetic Training Data in Dense Retrieval
- 用大模型根据查询和正例自动生成负样本
- 30B参数模型生成的负例反而降低检索性能
- 小模型如14B在部分数据集上表现最佳
训练高效的稠密检索模型通常依赖于从大型文档语料库中挖掘的硬负例（HN），常用方法如BM25或交叉编码器，但需完整语料库访问和昂贵的索引构建。本文提出直接利用大语言模型（LLM）从给定查询和正例段落生成合成硬负例。我们使用四个参数量从4B到30B的先进LLM（Qwen3、LLaMA3、Phi4）生成负例，并对DistilBERT进行微调，在10个BEIR基准数据集上评估性能。与普遍认为更强生成模型产生更好数据的假设相反，发现该生成流程在多数情况下仍逊于基于语料库的传统挖掘策略（BM25和交叉编码器）。此外，增大生成器规模并未单调提升检索性能，14B模型在多个任务中优于30B模型，甚至在某些设置下表现最差。
原文摘要 · Abstract (English)
Training effective dense retrieval models typically relies on hard negative (HN) examples mined from large document corpora using methods such as BM25 or cross-encoders, which require full corpus access and expensive index construction. We propose generating synthetic hard negatives directly from a provided query and positive passage, using Large Language Models(LLMs). We fine-tune DistilBERT using synthetic negatives generated by four state-of-the-art LLMs ranging from 4B to 30B parameters (Qwen3, LLaMA3, Phi4) and evaluate performance across 10 BEIR benchmark datasets. Contrary to the prevailing assumption that stronger generative models yield better synthetic data, find that our generative pipeline consistently underperforms traditional corpus-based mining strategies (BM25 and Cross-Encoder). Furthermore, we observe that scaling the generator model does not monotonically improve retrieval performance and find that the 14B parameter model outperforms the 30B model and in some settings it is the worst performing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。