让检索模型在隐空间内无声推理,兼顾准确与效率。
LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval
- 用双视角训练将显式推理融入隐空间,避免逐词生成
- 在多个基准上超越现有方法,小模型也表现稳健
- 适合需要强推理又怕慢的检索系统开发者
大语言模型已彻底改变密集检索,将主干从判别式编码器升级为生成式架构。然而关键断层仍存:尽管大模型具备强大推理能力,当前检索器多将其当作静态编码器,未充分挖掘其复杂推理潜力。现有方法通常采用重写-再检索流水线,在检索前生成显式思维链(CoT)理由,但带来高昂延迟。本文提出LaSER,一种新颖的自蒸馏框架,将显式推理内化到密集检索器的隐空间中。基于共享的LLM主干,LaSER引入双视角训练机制:显式视角显式编码真实推理路径,隐式视角进行隐式潜在思考。为弥合两视角差距,设计多粒度对齐策略,不仅对齐输出,更通过轨迹对齐机制同步隐式路径的中间潜在状态与显式推理段的语义进展。使检索器无需自回归生成即可沉默高效地思考。在领域内与领域外的推理密集型基准上,实验表明LaSER显著优于现有最先进方法。跨多种主干和模型规模的分析验证了方法鲁棒性,证实统一学习框架对激发有效隐式思考至关重要。本方法成功融合显式思维链管道的推理深度与标准密集检索器的推理效率。
原文摘要 · Abstract (English)
LLMs have fundamentally transformed dense retrieval, upgrading backbones from discriminative encoders to generative architectures. However, a critical disconnect remains: while LLMs possess strong reasoning capabilities, current retrievers predominantly utilize them as static encoders, leaving their potential for complex reasoning unexplored. To address this, existing approaches typically adopt rewrite-then-retrieve pipelines to generate explicit CoT rationales before retrieval. However, this incurs prohibitive latency. In this paper, we propose LaSER, a novel self-distillation framework that internalizes explicit reasoning into the latent space of dense retrievers. Operating on a shared LLM backbone, LaSER introduces a dual-view training mechanism: an Explicit view that explicitly encodes ground-truth reasoning paths, and a Latent view that performs implicit latent thinking. To bridge the gap between these views, we design a multi-grained alignment strategy. Beyond standard output alignment, we introduce a trajectory alignment mechanism that synchronizes the intermediate latent states of the latent path with the semantic progression of the explicit reasoning segments. This allows the retriever to think silently and effectively without autoregressive text generation. Extensive experiments on both in-domain and out-of-domain reasoning-intensive benchmarks demonstrate that LaSER significantly outperforms state-of-the-art baselines. Furthermore, analyses across diverse backbones and model scales validate the robustness of our approach, confirming that our unified learning framework is essential for eliciting effective latent thinking. Our method successfully combines the reasoning depth of explicit CoT pipelines with the inference efficiency of standard dense retrievers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。