让大模型并行拆解搜索任务,提升效率与准确率
ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning
- 用强化学习教会大模型识别可并行的查询结构
- 在7个基准上平均提升2.9%,并行问题上性能增12.7%
- 减少30.4%的调用次数,适合复杂多步检索场景
如Search-R1等增强推理的搜索代理,通过可验证奖励的强化学习(RLVR)训练,在从外部知识源进行多步信息检索方面表现出色。这些代理通过动态获取相关事实,克服了参数化记忆的局限。然而,现有方法存在根本性架构缺陷:即使查询中存在逻辑独立且可并行的子任务,仍严格按顺序处理,造成计算效率瓶颈,尤其在需多个实体比较的查询中更为显著。为此,我们提出ParallelSearch,一种新型强化学习框架,使大语言模型(LLMs)能够识别可并行的查询结构,并并发执行多个搜索操作。该方法引入专用奖励函数,激励模型识别独立查询组件,同时通过联合考虑正确性、分解质量与并行执行优势来保持答案准确性。全面实验表明,ParallelSearch在七个问答基准上平均优于现有最优基线2.9%。特别地,在可并行查询上,性能提升达12.7%,且仅需序列方法69.6%的LLM调用次数。
原文摘要 · Abstract (English)
Reasoning-augmented search agents such as Search-R1, trained via reinforcement learning with verifiable rewards (RLVR), demonstrate remarkable capabilities in multi-step information retrieval from external knowledge sources. These agents address the limitations of their parametric memory by dynamically gathering relevant facts to address complex reasoning tasks. However, existing approaches suffer from a fundamental architectural limitation: they process search queries strictly sequentially, even when handling inherently parallelizable and logically independent comparisons. This sequential bottleneck significantly constrains computational efficiency, particularly for queries that require multiple entity comparisons. To address this critical limitation, we propose ParallelSearch, a novel reinforcement learning framework that empowers large language models (LLMs) to recognize parallelizable query structures and execute multiple search operations concurrently. Our approach introduces dedicated reward functions that incentivize the identification of independent query components while preserving answer accuracy through jointly considering correctness, query decomposition quality, and parallel execution benefits. Comprehensive experiments demonstrate that ParallelSearch outperforms state-of-the-art baselines by an average performance gain of 2.9% across seven question-answering benchmarks. Notably, on parallelizable questions, our method achieves a 12.7% performance improvement while requiring only 69.6% of the LLM calls compared to sequential approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。