用大模型推理增强文本嵌入，提升零样本检索效果
Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval
- 生成中间推理文本，再计算嵌入，融入逻辑推理能力
- 在BRIGHT基准上显著提升零样本检索性能
- 适合需要深层推理的跨领域检索场景
基于Transformer的模型如BERT和E5通过捕捉丰富的上下文表示显著推进了文本嵌入技术。然而，许多现实世界查询需要复杂推理才能找到相关文档，仅依赖表面词法匹配的编码器型检索器常表现不足。解码器型大语言模型（LLMs）虽具备强大推理能力，但现有基于LLM的嵌入方法仍主要关注上下文表示，未充分挖掘其推理潜力。为此，我们提出推理增强型文本嵌入（RITE），一种简单而有效的方法，利用生成式LLM在标记空间中生成中间推理文本，再计算嵌入，从而在表示中注入推理深度。在强调推理能力的BRIGHT检索基准上的实验表明，RITE在多个领域显著提升了零样本检索性能，验证了将推理融入嵌入过程的有效性。
原文摘要 · Abstract (English)
Transformer-based models such as BERT and E5 have significantly advanced text embedding by capturing rich contextual representations. However, many complex real-world queries require sophisticated reasoning to retrieve relevant documents beyond surface-level lexical matching, where encoder-only retrievers often fall short. Decoder-only large language models (LLMs), known for their strong reasoning capabilities, offer a promising alternative. Despite this potential, existing LLM-based embedding methods primarily focus on contextual representation and do not fully exploit the reasoning strength of LLMs. To bridge this gap, we propose Reasoning-Infused Text Embedding (RITE), a simple but effective approach that integrates logical reasoning into the text embedding process using generative LLMs. RITE builds upon existing language model embedding techniques by generating intermediate reasoning texts in the token space before computing embeddings, thereby enriching representations with inferential depth. Experimental results on BRIGHT, a reasoning-intensive retrieval benchmark, demonstrate that RITE significantly enhances zero-shot retrieval performance across diverse domains, underscoring the effectiveness of incorporating reasoning into the embedding process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。