用合成计划引导研究型智能体探索工具，提升深度研究能力
SynPlanResearch-R1: Encouraging Tool Exploration for Deep Research with Synthetic Plans
- 通过合成工具使用轨迹增强冷启动阶段的探索行为
- 在7个多跳任务上相比最优基线提升最高6.0%
- 适合需要复杂推理与网络搜索的研究型AI系统开发者
研究型智能体可通过工具从网络获取信息来回答用户问题，需动态融合内部推理与工具调用。尽管理论上可通过可验证奖励的强化学习（RLVR）学习该能力，但我们发现智能体常表现出探索不足，如过早终止和工具使用偏倚。因此，仅靠RLVR改进有限。为此提出SynPlanResearch-R1框架，通过合成工具使用轨迹，在冷启动监督微调阶段引导更深入的探索，为后续强化学习提供良好初始化。在七个多跳与开放网络基准测试中，相较于现有最优基线，该框架在Qwen3-8B和Qwen3-4B模型上的性能分别提升最多6.0%和5.8%。对工具使用模式与训练动态的进一步分析揭示了性能提升的关键因素。代码已公开于https://github.com/HansiZeng/syn-plan-research。
原文摘要 · Abstract (English)
Research Agents enable models to gather information from the web using tools to answer user queries, requiring them to dynamically interleave internal reasoning with tool use. While such capabilities can in principle be learned via reinforcement learning with verifiable rewards (RLVR), we observe that agents often exhibit poor exploration behaviors, including premature termination and biased tool usage. As a result, RLVR alone yields limited improvements. We propose SynPlanResearch-R1, a framework that synthesizes tool-use trajectories that encourage deeper exploration to shape exploration during cold-start supervised fine-tuning, providing a strong initialization for subsequent RL. Across seven multi-hop and open-web benchmarks, \framework improves performance by up to 6.0% on Qwen3-8B and 5.8% on Qwen3-4B backbones respectively compared to SOTA baselines. Further analyses of tool-use patterns and training dynamics compared to baselines shed light on the factors underlying these gains. Our code is publicly available at https://github.com/HansiZeng/syn-plan-research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。