让模型推理过程更可信,提升密集检索准确率。
Retrieval Grounding Latent Reasoning for Dense Retrieval

- 用隐式推理轨迹显式建模中间思考步骤
- 在多个基准上超越强基线,且推理高效
- 适合需要可解释推理的检索系统
推理密集型检索要求文本表示不仅捕捉语义相似性,还需包含根据给定检索指令判断相关性的推理能力。现有增强推理的嵌入模型通过将推理信息融入稠密表示来提升检索效果,但其监督通常以最终检索目标为主导,导致潜在推理路径可能学习到捷径模式,维持检索性能却无法带来实质性提升。我们提出检索引导隐式推理(RGLT)框架,显式关联中间隐状态转移与检索改进。RGLT通过指令条件化的隐空间非自回归推理,利用静默标记构建推理轨迹;结合过程监督的显式到隐式蒸馏与检索引导监督,采用分阶段思维链重构来塑造中间隐状态,并使用检索效果信用分配优化整个推理轨迹中的增量收益。在多个推理密集型检索基准上的实验表明,RGLT持续优于强基线,同时保持高效的嵌入推理能力。
原文摘要 · Abstract (English)
Reasoning-intensive retrieval requires text representations to capture not only semantic similarity, but also the reasoning needed to determine relevance under a given retrieval instruction. Existing reasoning-enhanced embedding models improve retrieval by incorporating reasoning information into dense representations, yet their supervision is typically dominated by the final retrieval objective. As a result, latent reasoning trajectories may learn shortcut reasoning patterns that preserve retrieval performance without producing meaningful incremental retrieval gains. We propose Retrieval Grounding Latent Reasoning (RGLT), a latent reasoning framework for dense retrieval that explicitly connects intermediate latent transitions with retrieval improvements. RGLT performs non-autoregressive reasoning in hidden space through an instruction-conditioned latent reasoning trajectory constructed from silent tokens. It combines process-supervised explicit-to-implicit distillation with retrieval-grounded supervision, using stage-wise CoT reconstruction to shape intermediate latent states and retrieval-effect credit to optimize incremental retrieval gains across the latent reasoning trajectories. Experiments on reasoning-intensive retrieval benchmarks show that RGLT consistently outperforms strong baselines while preserving efficient embedding inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。