让检索模型先推理再生成嵌入，提升复杂查询理解能力
GEM: A Generative Embedding Model Bridging Reasoning and Retrieval
- 用同一模型先推理用户意图，再生成带上下文的检索嵌入
- 在复杂任务上超越不推理的基线，媲美更大模型性能
- 支持提示工程动态扩展计算量，适合需要精准理解的场景
现代大语言模型在推理和指令遵循方面表现优异，使用户能表达复杂多样的信息需求。然而，传统检索器主要依赖查询与文档间的表面匹配，导致用户表达方式与检索器理解之间差距日益扩大。本文提出GEM，一种生成式嵌入模型，通过自身知识显式推理用户意图和相关性标准，增强检索能力。GEM将生成与嵌入统一于单一模型：先对查询进行推理，再添加嵌入标记以编码丰富上下文用于检索。在推理密集型和指令遵循型检索任务上，GEM展现出其推理增强检索的有效性，优于无推理变体，并达到使用更大模型基线的水平。此外，GEM的生成特性支持通过提示工程在测试时扩展计算量，进一步提升检索性能。代码已公开于：https://anonymous.4open.science/r/GEM。
原文摘要 · Abstract (English)
Modern LLMs excel at reasoning and instruction following, enabling users to express complex and diverse information needs. However, conventional retrievers largely rely on surface-level matching between queries and documents, resulting in a growing gap between how users express their needs and how retrievers interpret them. In this paper, we present GEM, a generative embedding model that augments retrieval through its own knowledge by explicitly reasoning about user intent and relevance criteria. GEM unifies generation and embedding within a single model: it first reasons over the query, then appends an embedding token to encode the enriched context for retrieval. Evaluated on reasoning-intensive and instruction-following retrieval tasks, GEM demonstrates the effectiveness of its reasoning-augmented retrieval, outperforming its non-reasoning variant and matching baselines using substantially larger models. Furthermore, GEM's generative nature allows test-time compute scaling via prompting to further enhance retrieval performance. Our code is available at: https://anonymous.4open.science/r/GEM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。