提出迭代式研究框架,解决长时序任务中上下文过载问题。
IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling
- 采用基于MDP的迭代架构,定期重构工作空间,避免信息堆积
- 在6个基准上平均提升14.5个百分点,2048次交互下性能从3.5%升至42.5%
- 适用于训练智能体或作为提示策略,可提升前沿模型19.2个百分点
近期深度研究智能体在通过外部源动态推理实现自主知识构建方面展现出潜力。然而,现有方法依赖单一上下文范式,将所有信息累积于一个不断扩展的上下文窗口中,导致上下文过载与噪声污染,限制其在长时序任务中的表现。我们提出 extbf{IterResearch},一种新颖的迭代式深度研究范式,从交互扩展视角重新思考长时序研究。不同于线性上下文累积,我们采用受马尔可夫决策过程启发的架构,并结合战略性工作区重构。通过维护一个演进的报告作为记忆,并定期合成见解,该方法在任意探索深度下保持稳定的推理能力。为有效训练此范式,我们采用效率感知策略优化(EAPO),通过几何奖励折扣激励高效探索,并使用自适应降采样实现稳定分布式训练。大量实验表明,IterResearch 在六个基准上平均提升14.5个百分点,显著缩小与顶尖专有系统差距。尤为突出的是,该范式展现出前所未有的交互扩展能力,在2048次交互下性能从3.5%跃升至42.5%,并作为提示策略,使前沿模型在长时序任务上相较ReAct提升最高达19.2个百分点。这些发现使IterResearch 成为一种通用的长时序推理解决方案,既可作为训练智能体,也可作为前沿模型的有效提示策略。
原文摘要 · Abstract (English)
Recent advances in deep-research agents have shown promise for autonomous knowledge construction through dynamic reasoning over external sources. However, existing approaches rely on a mono-contextual paradigm that accumulates all information in a single, expanding context window, leading to context suffocation and noise contamination that limit their effectiveness on long-horizon tasks. We introduce \textbf{IterResearch}, a novel iterative deep-research paradigm that revisits long-horizon research through the lens of Interaction Scaling. Instead of relying on linear context accumulation, we adopt an MDP-inspired architecture with strategic workspace reconstruction. By maintaining an evolving report as memory and periodically synthesizing insights, our approach preserves consistent reasoning capacity across arbitrary exploration depths. To effectively train this paradigm, we employ Efficiency-Aware Policy Optimization (EAPO), a training strategy that adapts geometric reward discounting to incentivize efficient exploration and utilizes adaptive downsampling for stable distributed training. Extensive experiments demonstrate that IterResearch achieves substantial improvements over existing open-source agents with average +14.5pp across six benchmarks and narrows the gap with frontier proprietary systems. Remarkably, our paradigm exhibits unprecedented interaction scaling, extending to 2048 interactions with dramatic performance gains (from 3.5\% to 42.5\%), and serves as an effective prompting strategy, improving frontier models by up to 19.2pp over ReAct on long-horizon tasks. These findings position IterResearch as a versatile solution for long-horizon reasoning, effective both as a trained agent and as a prompting paradigm for frontier models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。