用大模型生成难负例会降低检索效果,本文提出新方法解决生成与判别间的差距。
When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrieval

- 通过思维链引导的反事实扰动构造可控的难负例
- 训练时最大化查询视角熵,避免模型依赖生成来源而非相关性
- 解决生成文本太自然导致对比学习失效的问题,适合检索系统优化者
硬负样本挖掘已成为训练检索器的主流策略,但存在固有局限:负样本受限于语料库范围,选择基于检索器得分而非诊断价值,且随着检索器性能提升,负样本污染日益严重。基于大语言模型(LLM)的合成提供了更优方案:负样本无约束、目标明确、无误报风险。然而我们发现,简单将生成负样本引入对比学习反而会损害检索性能。原因在于生成-判别差距:LLM生成优化的是流畅自然的文本,而对比学习需要在决策边界处刻意制造相关性违反。分析揭示两种恶化现象:判别无关生成,即LLM缺乏对查询信息需求的理解,生成通用或主题偏移文本,无法提供对比信号;源依赖捷径,即分布特征使模型能通过来源区分负样本,引发梯度漂移,破坏优化过程。为此,我们提出CausalNeg,包含两个模块:(1) 思维链引导的反事实扰动用于数据构建:将文档满足查询的原因分解为显式信息需求,再针对性地违反其中一项需求,生成具有可控、可解释难度的负样本;(2) 训练中查询视角熵最大化:使生成负样本在相似度谱上分散分布,最小化来源身份与相似度分数之间的互信息,抑制捷径利用。代码已开源于https://github.com/mzhangzhicheng/CausalNeg。
原文摘要 · Abstract (English)
Hard negative mining has become the dominant strategy for training retrievers, yet it faces intrinsic limitations: negatives are bounded by corpus availability, selected by retriever score rather than diagnostic value, and increasingly contaminated by false positives as the retriever improves. LLM-based synthesis offers a principled alternative, where negatives that are unconstrained, targeted, and free from false positive risk. But we show that naively incorporating generated negatives into contrastive learning often degrades retrieval performance. We identify and formalize the root cause as a generative-discriminative gap: LLM generation optimizes for fluent, plausible text, while contrastive learning demands strategic violations of relevance at the decision boundary. Our analysis reveals two compounding failure modes: discriminative-agnostic generation, where the LLM lacks an explicit model of query information needs and defaults to generic or topic-drifted text that provides no contrastive signal; and source-dependent shortcuts, where distributional artifacts enable the model to distinguish negatives by origin rather than relevance, causing gradient drift that actively corrupts optimization. To close this gap, we propose CausalNeg consisting of two main modules: (1) CoT-guided counterfactual perturbation for data construction: decomposes why a document satisfies a query into explicit information requirements, then surgically violates individual requirements to construct negatives with controlled, interpretable hardness. (2) Query-view entropy maximization during training: disperses generated negatives across the similarity spectrum, minimizing the mutual information between source identity and similarity scores to suppress shortcut exploitation. We make our code publicly available at https://github.com/mzhangzhicheng/CausalNeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。