arXiv:2510.08276cs.CL2025-10被引 22

让大模型在长对话中持续深思,突破上下文长度限制。

Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window

  • 用真实网页构造复杂可验证的问答对,提升训练难度与可靠性。
  • 动态上下文管理使模型在32k上下文内稳定运行近100轮对话,准确率达33.5%。
  • 适合研究长程推理、多轮交互或搜索智能体的开发者与研究员。

尽管近期推理模型通过强化学习展现了认知行为,现有方法在长周期多轮交互中仍难以激发深层推理能力。本文提出DeepMiner框架,通过引入高难度训练任务和动态上下文窗口机制,实现这一目标。DeepMiner采用逆向构建法,从真实网络来源生成复杂且可验证的问答对,确保训练数据的挑战性与可靠性,同时注入多轮推理能力。进一步设计了简洁高效的动态上下文管理策略,在训练与推理中均采用滑动窗口机制,无需依赖外部摘要模型,有效应对持续扩展的长时序上下文。基于Qwen3-32B进行强化学习,我们训练出DeepMiner-32B,在多个搜索代理基准上取得显著提升:在BrowseComp-en上达到33.5%准确率,较此前最佳开源模型高出近20个百分点,并在BrowseComp-zh、XBench-DeepSearch和GAIA上保持一致改进。值得注意的是,该动态上下文管理策略支持标准32k上下文长度下近100轮的持续交互,有效解决了现有系统因上下文限制导致的瓶颈问题。

原文摘要 · Abstract (English)

While recent advances in reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep reasoning capabilities in multi-turn agents with long-horizon interactions. We propose DeepMiner, a novel framework that elicits such abilities by introducing high-difficulty training tasks and dynamic context window. DeepMiner presents a reverse construction method to generate complex but verifiable question-answer pairs from authentic web sources, which ensures the challenge and reliability of training data while injecting cognitive capabilities into multi-turn reasoning scenarios. We further design an elegant yet effective dynamic context management strategy for both training and inference, utilizing sliding window mechanisms while eliminating the dependency on external summarization models, thereby efficiently empowering the model to handle continuously expanding long-horizon contexts. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source agent by almost 20 percentage points, and demonstrates consistent improvements on BrowseComp-zh, XBench-DeepSearch, and GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within standard 32k context length, effectively addressing the context limitations that constrain existing multi-turn interaction systems.

多轮推理长上下文搜索代理强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。