arXiv:2605.00353cs.IR2026-05

通过智能负样本挖掘提升宜家搜索召回效果,但线上表现受用户零点击行为制约。

Negative Data Mining for Contrastive Learning in Dense Retrieval at IKEA.com

论文配图:Negative Data Mining for Contrastive Learning in Dense Retrieval at IKEA.com
图 1 · 摘自论文原文
  • 基于商品层级与属性设计语义挑战性负样本,增强对比学习质量。
  • 用大模型评分所有候选商品,生成高质量训练数据,离线准确率提升2.6%。
  • 发现67%热门搜索无点击,提示需结合真实用户行为优化评估体系。

对比学习是现代检索系统的核心,其效果高度依赖训练时负例的质量。本文针对宜家产品搜索,提出系统性改进方法:一是利用商品层级分类和属性信息设计结构化负样本采样策略,生成语义上具有挑战性的负例;二是构建基于大模型的评估体系,为每个查询下的所有候选商品打分,替代稀疏人工标注或随机采样。该方法在加拿大市场离线实验中实现平均类别准确率提升2.6%。然而,对长尾查询的A/B测试显示,改进模型与基线在用户参与度指标上无显著差异(p > 0.05)。分析表明,67%的热门搜索零点击率超过50%,说明大量搜索会直接结束而未产生任何产品互动。这凸显了硬负样本挖掘的重要性,也强调训练数据与离线评估必须反映真实用户搜索行为——包括查询意图分布与零点击模式——才能弥合离线检索性能与线上用户体验之间的差距。

原文摘要 · Abstract (English)

Contrastive learning is a core component of modern retrieval systems, but its effectiveness heavily relies on the quality of negative examples used during training. In this work, we present a systematic approach to improving dense retrieval for IKEA product search through structured negative sampling strategies and scalable LLM-as-a-judge relevance evaluation. Building on IKEA Search Engine's late-interaction retrieval architectures, we introduce two key contributions: (1) structured negative sampling strategies that leverage product hierarchical taxonomy and product attributes to generate semantically challenging negatives, and (2) a comprehensive LLM-based evaluation methodology for generating training data. Rather than relying on sparse human annotations or random sampling, our LLM-based evaluation system allocates a score for all candidate products against each query. Our methodology achieves +2.6\% average category accuracy on offline real user query experiments on the Canada market. However, our A/B test on long-tail queries showed no statistically significant differences in user engagement metrics between the improved and baseline models ($p > 0.05$). We trace this gap to user search behavior: 67\% of popular searches exhibit zero-click rates above 50\%, indicating that a substantial proportion of search sessions result in no product engagement regardless of result ranking. These findings underscore the importance of hard negative mining but also the need for grounding training data and offline evals in real user search behavior -- including query intent distribution and zero-click patterns -- to bridge the gap between offline retrieval quality and online user engagement.

对比学习负样本挖掘检索系统零点击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。