arXiv:2606.16316cs.IRcs.AI2026-06被引 1

用强化学习优化知识索引,让检索更快更准。

RL-Index: Reinforcement Learning for Retrieval Index Reasoning

论文配图:RL-Index: Reinforcement Learning for Retrieval Index Reasoning
图 1 · 摘自论文原文
  • 把推理过程提前到索引阶段,用大模型生成显式推理链
  • 在BRIGHT数据集上提升检索与问答性能,线上延迟大幅降低
  • 通用性强,可适配多种检索器和生成器,即插即用

外部知识检索对现实任务至关重要,但当查询与相关知识通过隐式推理(如共用定理或代码逻辑)关联时,现有方法难以应对。当前方法主要依赖查询端推理,导致在线延迟高,且未充分利用知识库中的推理语义。本文提出RL-Index,将检索索引推理建模为强化学习问题。通过在文档中添加由大模型生成的推理链,显式编码潜在的查询-知识关联,将推理移至索引阶段。采用组相对策略优化(GRPO),以检索相似性作为代理奖励信号,直接优化索引决策以提升检索效果。在BRIGHT基准上的大量实验表明,RL-Index持续提升检索与下游问答性能,显著降低在线推理延迟。所学推理增强具备跨不同检索器与生成器的泛化能力,验证了其作为即插即用索引策略的鲁棒性。

原文摘要 · Abstract (English)

Retrieving external knowledge is crucial for real-world tasks but remains difficult when queries and relevant knowledge are linked by implicit reasoning (e.g., shared theorems or coding logic). Existing methods rely mainly on query-side reasoning, leading to high online latency and underutilizing the reasoning semantics within the knowledge corpus. In this paper, we propose $\textbf{RL-Index}$, an indexing framework that formulates retrieval index reasoning as a reinforcement learning problem. Instead of performing reasoning at query time, RL-Index shifts reasoning to the indexing stage by augmenting documents with LLM-generated rationales that explicitly encode the latent query-knowledge relationship. To optimize the quality of these rationales, we employ Group Relative Policy Optimization (GRPO) and use retrieval similarity as a proxy reward signal, enabling direct optimization of indexing decisions for retrieval effectiveness. Extensive experiments on the BRIGHT benchmark demonstrate that RL-Index consistently improves both retrieval and downstream question-answering performance, while significantly reducing online inference latency. Moreover, the learned rationale augmentation generalizes across diverse retrievers and generators, highlighting its robustness as a plug-and-play indexing strategy across different retrieval systems.

检索增强强化学习索引优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。