让检索增强生成系统各模块协同优化，提升效果并减少检索开销
SmartRAG: Jointly Learn RAG-Related Tasks From the Environment Feedback
- 用强化学习统一训练检索、重写和生成模块
- 联合优化后性能优于独立训练，检索次数减少
- 适合需要高效精准问答的智能系统研发者
RAG 系统由多个模块协同工作，但这些模块通常各自独立训练。我们提出，像 RAG 这样包含多个模块的系统应进行联合优化以达到最佳性能。为此，我们设计了名为 SmartRAG 的特定流程，包含一个策略网络和一个检索器。该策略网络可作为决策者决定何时检索、重写查询以适配检索器，以及生成最终回答（有/无观察结果）。我们采用强化学习算法联合优化整个系统，奖励机制旨在促使系统在最小化检索成本的前提下实现最优表现。联合优化使各模块能感知彼此工作状态，从而找到整体最优协作方式。实验表明，联合优化的 SmartRAG 性能优于独立优化的基准模型。
原文摘要 · Abstract (English)
RAG systems consist of multiple modules to work together. However, these modules are usually separately trained. We argue that a system like RAG that incorporates multiple modules should be jointly optimized to achieve optimal performance. To demonstrate this, we design a specific pipeline called \textbf{SmartRAG} that includes a policy network and a retriever. The policy network can serve as 1) a decision maker that decides when to retrieve, 2) a query rewriter to generate a query most suited to the retriever, and 3) an answer generator that produces the final response with/without the observations. We then propose to jointly optimize the whole system using a reinforcement learning algorithm, with the reward designed to encourage the system to achieve the best performance with minimal retrieval cost. When jointly optimized, all the modules can be aware of how other modules are working and thus find the best way to work together as a complete system. Empirical results demonstrate that the jointly optimized SmartRAG can achieve better performance than separately optimized counterparts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。