用隐式细粒度思维提升大模型检索效率与准确性
SHIFT: Self-reconstruction Harnesses Implicit Fine-grained Thinking for Retrieval
- 通过残差投影和双向注意力聚合，将大模型转化为高效推理检索器
- 采用细粒度下一轮词预测重建，缓解检索与生成目标不匹配问题
- 在复杂推理检索任务中表现优于主流方法，适合高精度信息检索场景
基于大模型的检索器已成为现代信息检索系统的核心组件。现有‘重写后检索’范式通过显式推理提升效果，而像 GIRCSE、LaSER 这类隐式推理检索器则用软标记替代显式推理以提高效率。尽管这些方法在需要推理的检索基准上表现良好，但依然难以解决检索与生成目标之间的不匹配问题。本文提出 SHIFT（Self-reconstruction Harnesses Implicit Fine-grained Thinking for Retrieval），一种基于大模型的检索训练框架。首先，在隐空间中通过残差投影和面向任务的双向注意力聚合，将大模型转换为高效推理检索器；其次，利用基于细粒度下一轮词预测的重建机制，缓解对比学习与隐式推理间的不一致。大量实验表明，SHIFT 在多个需要推理的检索基准上持续优于其他主流检索器，并进行了详细分析以揭示其工作原理。
原文摘要 · Abstract (English)
LLM-based retrievers have become a fundamental component of modern information retrieval systems. The paradigm of "rewrite-then-retriev" introduces explicit reasoning before retrieval. In addition, implicit-reasoning retrievers such as GIRCSE and LaSER improve efficiency by replacing explicit reasoning with soft tokens. Although these methods demonstrated competitive performance on reasoning-intensive retrieval benchmarks, they struggle to address the mismatch between the objectives of retrieval and generation. In this work, we propose SHIFT ($\textbf{S}$elf-reconstruction $\textbf{H}$arnesses $\textbf{I}$mplicit $\textbf{F}$ine-grained $\textbf{T}$hinking for Retrieval), a retrieval training framework based on LLMs. Firstly, we transfer LLMs into reasoning-efficient retrievers with residual projection and task-oriented bidirectional attention aggregation in the latent space. Secondly, we alleviate the mismatch between contrastive learning and implicit reasoning using fine-grained next-token-prediction-based reconstruction. Extensive experiments on reasoning-intensive retrieval benchmarks show that SHIFT consistently outperforms other widely used retrievers. We also carried out a detailed analysis to illustrate how our method works.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。