arXiv:2605.06285cs.CLcs.LG2026-05

将复杂问答的推理与检索从语言空间转到隐空间,大幅降低延迟。

LatentRAG: Latent Reasoning and Retrieval for Efficient Agentic RAG

论文配图:LatentRAG: Latent Reasoning and Retrieval for Efficient Agentic RAG
图 1 · 摘自论文原文
  • 用隐空间直接生成思考和子查询,单次前向传播完成
  • 在7个数据集上性能接近显式方法,推理延迟降低约90%
  • 支持端到端优化,且能将隐表示解码为自然语言以增强可解释性

单步检索增强生成(RAG)适用于简单问答,但难以处理复杂问题。代理型RAG通过多步迭代过程,让大语言模型(LLM)生成中间思考和子查询,与检索系统交互。然而,这种自回归生成长文本思考和子查询导致显著延迟。为此,我们提出LatentRAG,将推理与检索从离散语言空间转移到连续隐空间。不同于现有显式方法逐词生成自然语言思考或子查询,LatentRAG通过一次前向传播直接从隐藏状态生成隐空间令牌。我们对齐LLM与密集检索模型的隐空间,实现对隐子查询令牌的检索,并支持端到端联合优化。为提升透明度并促进语义有意义的隐表示,引入并行隐解码机制,将隐令牌还原为自然语言。在七个基准数据集上的实验表明,LatentRAG性能与显式代理RAG相当,同时推理延迟降低约90%,显著缩小与传统单步RAG的延迟差距。

原文摘要 · Abstract (English)

Single-step retrieval-augmented generation (RAG) provides an efficient way to incorporate external information for simple question answering tasks but struggles with complex questions. Agentic RAG extends this paradigm by replacing single-step retrieval with a multi-step process, in which the large language model (LLM) acts as a search agent that generates intermediate thoughts and subqueries to iteratively interact with the retrieval system. This iterative process incurs substantial latency due to the autoregressive generation of lengthy thoughts and subqueries. To address this limitation, we propose LatentRAG, a novel framework that shifts both reasoning and retrieval from discrete language space to continuous latent space. Unlike existing explicit methods that generate natural language thoughts or subqueries token-by-token, LatentRAG produces latent tokens for thoughts and subqueries directly from the hidden states in a single forward pass. We align LLMs with dense retrieval models in the latent space, enabling retrieval over latent subquery tokens and supporting end-to-end joint optimization. To improve transparency and encourage semantically meaningful latent representations, we incorporate a parallel latent decoding mechanism that translates latent tokens back into natural language. Extensive experiments on seven benchmark datasets show that LatentRAG achieves performance comparable to explicit agentic RAG methods while reducing inference latency by approximately 90%, substantially narrowing the latency gap with traditional single-step RAG.

RAG推理加速隐空间大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。