将规划与搜索解耦，提升智能体检索生成系统的效率与准确性。
DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling
- 用双价值模型分离规划与搜索，支持独立优化
- 在不同规模模型上验证，显著提升任务成功率
- 适合需要精准推理与高效搜索的复杂任务场景
检索增强生成（RAG）系统通过动态整合外部知识，成为提升大语言模型性能的关键方法。为增强RAG的灵活性，代理式RAG引入了自主代理。然而，代理式RAG面临三大挑战：（1）每一步的成功依赖高质量的规划与精确的搜索；（2）中间推理步骤缺乏监督信号；（3）规划与搜索的候选空间呈指数级增长。为此，我们提出DecoupleSearch框架，通过双价值模型解耦规划与搜索过程，实现计划推理与搜索落地的独立优化。该方法构建推理树，每个节点代表一个规划或搜索步骤，并利用蒙特卡洛树搜索评估每一步质量。推理时，分层束搜索结合双价值模型迭代优化计划与搜索候选。在多种参数规模的策略模型上进行的大量实验表明，该方法有效提升了系统性能。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs) through the dynamic integration of external knowledge. To further improve RAG's flexibility, Agentic RAG introduces autonomous agents into the workflow. However, Agentic RAG faces several challenges: (1) the success of each step depends on both high-quality planning and accurate search, (2) the lack of supervision for intermediate reasoning steps, and (3) the exponentially large candidate space for planning and searching. To address these challenges, we propose DecoupleSearch, a novel framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. Our approach constructs a reasoning tree, where each node represents planning and search steps. We leverage Monte Carlo Tree Search to assess the quality of each step. During inference, Hierarchical Beam Search iteratively refines planning and search candidates with dual value models. Extensive experiments across policy models of varying parameter sizes demonstrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。