让大模型学会动态调整搜索强度，更聪明地查互联网。
DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning
- 用强化学习训练模型根据问题难度自动调节搜索频率和深度。
- 在真实网络任务上，小模型性能接近6710亿参数的大模型。
- 适用于需要深度推理的开放问题，如长文生成与复杂问答。
信息检索需要迭代式证据收集与反思性推理，但当前大语言模型在开放网络问答中仍表现不佳。现有提示工程与监督微调方法受限于固定规则或训练语料，且通常仅在结构化维基数据上评估，缺乏现实适应性。我们提出 WebPuzzle，一个包含24,000个训练样本和275个测试样本的基准，用于评估真实互联网上的信息检索能力，涵盖维基与开放域查询。基于7,000个WebPuzzle实例，我们构建了DeepDiver——一种强化学习框架，催生出“搜索强度自适应”（Search Intensity Scaling, SIS）这一新兴能力：模型能主动提升搜索频率与深度，而非过早给出缺乏证据的答案。借助SIS，Qwen2.5-7B-Instruct与Pangu-7B-Reasoner在真实网络任务上的表现达到6710亿参数的DeepSeek-R1水平。我们详细阐述从冷启动微调到精心设计的强化学习流程的训练路径，并证明其搜索策略可从封闭式问题泛化至开放式生成任务，如长文本写作。本研究推动了大模型的自适应信息检索能力，并为未来工作提供了严谨的评估基准。
原文摘要 · Abstract (English)
Information seeking demands iterative evidence gathering and reflective reasoning, yet large language models (LLMs) still struggle with it in open-web question answering. Existing prompting and supervised fine-tuning (SFT) methods remain fixed by prompt rules or training corpora, and are usually benchmarked only on well-structured wiki sources, limiting real-world adaptability. We introduce WebPuzzle, a 24k-sample training and 275-sample test benchmark that evaluates information seeking on the live internet, across both wiki and open-domain queries. Leveraging 7k WebPuzzle instances, we develop DeepDiver, a reinforcement-learning (RL) framework that cultivates Search Intensity Scaling (SIS)-an emergent ability to escalate search frequency and depth instead of settling on overconfident, under-evidenced answers. With SIS, Qwen2.5-7B-Instruct and Pangu-7B-Reasoner attain performance on real-web tasks comparable to the 671B-parameter DeepSeek-R1. We detail DeepDiver's curriculum from cold-start SFT to a well designed RL procedure, and show that its seeking policy generalized from closed-ended queries to open-ended generation such as long-form writing. Our results advance adaptive information seeking in LLMs and provide a rigorous benchmark for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。