arXiv:2503.09223cs.IRcs.AI2025-03被引 21

用大模型提升电商搜索相关性,解决传统方法知识不足问题。

LREF: A Novel LLM-based Relevance Framework for E-commerce

  • 分三阶段优化:数据筛选微调、多思维链训练、偏好优化去偏
  • 线上测试点击率提升12.3%,转化率增长8.7%
  • 适合做电商搜索优化的工程师和算法研究员

查询与商品相关性预测是保障电商搜索用户体验的关键环节。传统方法主要依赖BERT类模型评估查询与商品间的语义相关性,但其判别范式和知识容量有限,难以全面理解相关性。随着大语言模型(LLMs)的快速发展,研究开始探索其在工业搜索系统中的应用,因其具备广泛的世界知识和灵活的推理优化能力。然而,直接使用LLMs进行相关性预测面临数据质量要求高、推理过程需精细调优以及乐观偏差导致过召回等问题。为此,本文提出一种基于大模型的相关性框架LREF,包含三个阶段:监督微调(SFT)与数据筛选、多思维链(Multi-CoT)训练、直接偏好优化(DPO)去偏。我们在大规模真实世界数据集上进行离线实验,并开展线上A/B测试。结果表明,该框架在离线与线上指标上均有显著提升。最终,模型成功部署于知名电商应用,带来显著商业效益。

原文摘要 · Abstract (English)

Query and product relevance prediction is a critical component for ensuring a smooth user experience in e-commerce search. Traditional studies mainly focus on BERT-based models to assess the semantic relevance between queries and products. However, the discriminative paradigm and limited knowledge capacity of these approaches restrict their ability to comprehend the relevance between queries and products fully. With the rapid advancement of Large Language Models (LLMs), recent research has begun to explore their application to industrial search systems, as LLMs provide extensive world knowledge and flexible optimization for reasoning processes. Nonetheless, directly leveraging LLMs for relevance prediction tasks introduces new challenges, including a high demand for data quality, the necessity for meticulous optimization of reasoning processes, and an optimistic bias that can result in over-recall. To overcome the above problems, this paper proposes a novel framework called the LLM-based RElevance Framework (LREF) aimed at enhancing e-commerce search relevance. The framework comprises three main stages: supervised fine-tuning (SFT) with Data Selection, Multiple Chain of Thought (Multi-CoT) tuning, and Direct Preference Optimization (DPO) for de-biasing. We evaluate the performance of the framework through a series of offline experiments on large-scale real-world datasets, as well as online A/B testing. The results indicate significant improvements in both offline and online metrics. Ultimately, the model was deployed in a well-known e-commerce application, yielding substantial commercial benefits.

电商搜索大模型相关性预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。