arXiv:2510.14321cs.IR2025-10被引 9

让搜索模型先推理再匹配,提升复杂查询的精准度。

Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigm

  • 引入推理过程生成增强查询嵌入,深化语义理解。
  • 在电商搜索中,复杂查询召回率提升18.7%。
  • 适合需要深度理解用户意图的智能搜索场景。

在现代电商搜索系统中,稠密检索已成为关键组件。通过计算查询与商品嵌入之间的相似性,可从大规模库存中高效筛选候选商品。随着大语言模型(LLM)的发展,主流嵌入模型逐渐从BERT转向基于LLM的架构,以实现更精确的文本建模。然而,这些模型仍采用直接嵌入方法,嵌入的语义准确性不足,因此大量依赖对比学习来对齐正样本对的语义。这导致模型倾向于捕捉训练数据中的统计共现模式,偏向浅层词法和语义匹配,对词汇差异显著的困难查询性能显著下降。为此,本文提出大型推理嵌入模型(LREM),创新性地将推理过程融入表示学习。对于困难查询,LREM首先进行推理以深入理解原始查询,再生成推理增强的查询嵌入用于检索。该推理过程有效弥合了原始查询与目标商品间的语义鸿沟,显著提升检索准确率。具体而言,我们采用两阶段训练:第一阶段在精心构建的查询-思维链-商品三元组上,通过监督微调(SFT)和InfoNCE损失优化LLM,建立初步推理与嵌入能力;第二阶段通过强化学习(RL)进一步精炼推理轨迹。大量离线与在线实验验证了LREM的有效性,其已自2025年8月起部署于中国最大电商平台。

原文摘要 · Abstract (English)

In modern e-commerce search systems, dense retrieval has become an indispensable component. By computing similarities between query and item (product) embeddings, it efficiently selects candidate products from large-scale repositories. With the breakthroughs in large language models (LLMs), mainstream embedding models have gradually shifted from BERT to LLMs for more accurate text modeling. However, these models still adopt direct-embedding methods, and the semantic accuracy of embeddings remains inadequate. Therefore, contrastive learning is heavily employed to achieve tight semantic alignment between positive pairs. Consequently, such models tend to capture statistical co-occurrence patterns in the training data, biasing them toward shallow lexical and semantic matches. For difficult queries exhibiting notable lexical disparity from target items, the performance degrades significantly. In this work, we propose the Large Reasoning Embedding Model (LREM), which novelly integrates reasoning processes into representation learning. For difficult queries, LREM first conducts reasoning to achieve a deep understanding of the original query, and then produces a reasoning-augmented query embedding for retrieval. This reasoning process effectively bridges the semantic gap between original queries and target items, significantly improving retrieval accuracy. Specifically, we adopt a two-stage training process: the first stage optimizes the LLM on carefully curated Query-CoT-Item triplets with SFT and InfoNCE losses to establish preliminary reasoning and embedding capabilities, and the second stage further refines the reasoning trajectories via reinforcement learning (RL). Extensive offline and online experiments validate the effectiveness of LREM, leading to its deployment on China's largest e-commerce platform since August 2025.

稠密检索推理增强电商搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。