arXiv:2607.21461cs.AI2026-07

让研究型智能体自我迭代优化,逐步逼近复杂问题的正确答案。

AREX: Towards a Recursively Self-Improving Agent for Deep Research

论文配图:AREX: Towards a Recursively Self-Improving Agent for Deep Research
图 1 · 摘自论文原文
  • 通过内外循环交替,先找证据再逐项验证并针对性改进。
  • 在多个基准上表现超越同规模模型,接近超大模型性能。
  • 适合需要长期推理和多步纠错的深度研究任务。

深度研究需智能体找到同时满足多重约束的答案,发现成本高但验证可分解为可处理的约束检查。这种发现-验证不对称性表明,研究智能体不应仅延长搜索,而应递归改进现有答案:通过验证中间结果,利用部分验证状态指导后续优化。我们提出AREX,一种递归自提升(RSI)深度研究智能体家族。AREX交替执行内层研究循环(收集证据、构建暂定答案)与外层自提升循环(逐约束审计答案、识别未解命题并发起针对性后续研究)。为实现长期递归自提升,AREX学习一个自主上下文更新工具,将不断增长的交互历史压缩为紧凑的改进状态,保留已验证证据与未决约束,无需依赖外部模型。我们在经验证的合成任务和高质量轨迹上,通过代理式中段训练与长程强化学习训练AREX。为缓解长周期学习中的稀疏最终奖励,我们强调关键步骤——如获得决定性证据或纠正错误研究方向。我们实现了40亿参数的密集模型和1220亿参数(100亿激活)的专家混合模型。在BrowseComp、WideSearch、DeepSearchQA、Humanity's Last Exam(HLE)等推理与工具使用基准上,AREX显著优于同等规模基线,且保持与大幅激活参数量模型相当的竞争力。

原文摘要 · Abstract (English)

Deep research requires agents to find answers that jointly satisfy multiple constraints. Discovering such answers is costly, whereas verifying a candidate can often be decomposed into tractable constraint-wise checks. This discovery--verification asymmetry suggests that a research agent should do more than simply search longer: it should recursively improve its current answer by verifying intermediate results and using the partially verified state to guide subsequent refinement. We introduce AREX, a family of Recursively Self-Improving (RSI) deep research agents. AREX alternates between an inner research loop that gathers evidence and constructs a provisional answer, and an outer self-improvement loop that audits the answer constraint-wise, identifies unresolved claims, and launches targeted follow-up research. To sustain RSI over long horizons, AREX learns an autonomous context-update tool that compresses growing interaction history into a compact improvement state preserving verified evidence and unresolved constraints, without relying on an external model. We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning. To mitigate sparse final rewards during long horizon learning, we emphasize key steps where decisive evidence is acquired or erroneous research directions are corrected. We instantiate a dense 4B model and a 122B-A10B Mixture-of-Experts model. Across BrowseComp, WideSearch, DeepSearchQA, Humanity's Last Exam (HLE), and other reasoning and tool-use benchmarks, AREX substantially outperforms comparable-scale baselines and remains competitive with models using substantially more activated parameters.

智能体自提升深度研究推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。