用连续潜空间统一检索与生成，提升问答准确率
CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning
- 在共享连续空间中联合优化检索与生成，避免模块割裂
- 压缩率16时仍优于现有方法，关键指标提升3.2个点
- 适合需要高效知识增强的生成任务开发者
检索增强生成（RAG）虽能提升大语言模型的外部知识利用能力，但仍面临上下文过长和检索-生成优化分离的问题。本文提出CLaRa（连续潜空间推理），通过基于嵌入的压缩和共享连续空间中的联合优化，实现端到端训练。为获得语义丰富且可检索的压缩向量，引入SCP——一种基于问答与改写监督的关键信息保留数据合成框架。CLaRa通过单语言建模损失端到端训练重排序器与生成器，并使用可微分top-k估计器使梯度贯穿两个模块。理论上，该统一优化使检索相关性与答案质量对齐。在多个问答基准测试中，即使压缩率达16，性能仍优于文本微调基线，达到当前最优。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but still suffers from long contexts and disjoint retrieval-generation optimization. In this work, we propose CLaRa (Continuous Latent Reasoning), a unified framework that performs embedding-based compression and joint optimization in a shared continuous space. To obtain semantically rich and retrievable compressed vectors, thereby reducing the document length fed into the generator, we introduce SCP, a key-preserving data synthesis framework based on question answering and paraphrase supervision. CLaRa then trains the reranker and generator end-to-end via a single language modeling loss, with gradients flowing through both modules using a differentiable top-k estimator. Theoretically, this unified optimization aligns retrieval relevance with answer quality. Experiments across multiple QA benchmarks show that CLaRa achieves state-of-the-art compression and reranking performance, even at a text compression rate of 16, outperforming text-based fine-tuned baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。