arXiv:2602.14234cs.AIcs.CL2026-02被引 33

提升长时序搜索智能体性能,解决高成本与稀疏奖励难题

REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents

  • 通过图结构与证据分布控制任务难度,实现可扩展的高质量任务生成
  • 引入工具增强查询,推动主动调用工具而非被动记忆,提升搜索效率
  • 构建本地模拟环境,支持低成本快速迭代强化学习实验

大语言模型正从通用知识引擎转向真实问题求解者,但优化其在深度搜索任务中的表现仍具挑战。核心瓶颈在于高质量搜索轨迹与奖励信号极度稀疏,源于长时序任务构建难以扩展,以及涉及外部工具调用的交互密集型回放成本高昂。为此,我们提出REDSearcher,一个统一框架,协同设计复杂任务合成、中期训练与后期训练,实现可扩展的搜索智能体优化。具体改进包括:(1) 将任务合成建模为双约束优化,通过图拓扑与证据分布精确控制任务难度,支持大规模生成高质量复杂任务;(2) 引入工具增强查询,促进主动使用工具而非被动回忆;(3) 中期训练中强化核心原子能力(知识、规划、函数调用),显著降低下游训练所需高质量轨迹的收集成本;(4) 构建本地模拟环境,实现快速、低成本的强化学习算法迭代。在纯文本与多模态搜索智能体基准上,本方法达到当前最优性能。为推动未来研究,我们将发布10,000条高质量复杂文本搜索轨迹、5,000条多模态轨迹及1,000条文本强化学习查询集,并提供代码与模型检查点。

原文摘要 · Abstract (English)

Large language models are transitioning from generalpurpose knowledge engines to realworld problem solvers, yet optimizing them for deep search tasks remains challenging. The central bottleneck lies in the extreme sparsity of highquality search trajectories and reward signals, arising from the difficulty of scalable longhorizon task construction and the high cost of interactionheavy rollouts involving external tool calls. To address these challenges, we propose REDSearcher, a unified framework that codesigns complex task synthesis, midtraining, and posttraining for scalable searchagent optimization. Specifically, REDSearcher introduces the following improvements: (1) We frame task synthesis as a dualconstrained optimization, where task difficulty is precisely governed by graph topology and evidence dispersion, allowing scalable generation of complex, highquality tasks. (2) We introduce toolaugmented queries to encourage proactive tool use rather than passive recall.(3) During midtraining, we strengthen core atomic capabilities knowledge, planning, and function calling substantially reducing the cost of collecting highquality trajectories for downstream training. (4) We build a local simulated environment that enables rapid, lowcost algorithmic iteration for reinforcement learning experiments. Across both textonly and multimodal searchagent benchmarks, our approach achieves stateoftheart performance. To facilitate future research on longhorizon search agents, we will release 10K highquality complex text search trajectories, 5K multimodal trajectories and 1K text RL query set, and together with code and model checkpoints.

搜索代理强化学习任务合成高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。