arXiv:2603.26499cs.AI2026-03被引 2

解决AI研究代理的三大性能瓶颈,提升长期探索能力

AIRA_2: Overcoming Bottlenecks in AI Research Agents

  • 采用异步多卡并行与动态反应式智能体,突破单卡瓶颈
  • 在MLE-bench-30上72小时达83.1%排名,超越最强基线72.7%
  • 适合追求高效长周期科研自动化的研究者

现有研究识别出AI研究代理的三大结构性瓶颈:(1) 同步单卡执行限制样本吞吐量,削弱搜索效益;(2) 基于验证集的选择导致泛化差距,性能随搜索时长下降;(3) 固定单轮大模型操作员限制搜索上限。本文提出AIRA$_2$,通过三项架构改进:异步多卡工作池实现吞吐量线性提升;隐藏一致评估协议提供可靠评价信号;ReAct智能体动态规划动作并交互调试。在MLE-bench-30上,AIRA$^{ iny extdagger}_{2}$ 24小时平均分位数为81.5%,72小时达83.1%,优于最强基线72.7%。在AIRS-Bench上,6/20项任务超过人类基准。消融实验表明各组件均必要,性能符合可迁移的可预测缩放规律,且先前报告的“过拟合”实由评估噪声引发,非真实记忆。

原文摘要 · Abstract (English)

Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the benefit of search; (2) a generalization gap where validation-based selection causes overfitting and performance to degrade over extended search horizons; and (3) the limited capability of fixed, single-turn LLM operators imposes a ceiling on search performance. We introduce AIRA$_2$, which addresses these bottlenecks through three architectural choices: an asynchronous multi-GPU worker pool that increases experiment throughput linearly; a Hidden Consistent Evaluation protocol that delivers a reliable evaluation signal; and ReAct agents that dynamically scope their actions and debug interactively. On MLE-bench-30, AIRA$^{\dagger}_{2}$ achieves a mean Percentile Rank of 81.5% at 24 hours and 83.1% at 72 hours, outperforming the strongest baseline, which achieves 72.7%. On AIRS-Bench, AIRA$_2$ exceeds human state-of-the-art on 6 out of 20 diverse research tasks. Ablations confirm that each architectural component is necessary, that performance follows a predictable scaling law that transfers across LLM backbones, and that the "overfitting" reported in prior work was driven by evaluation noise rather than true data memorization.

AI研究代理自动化科研ReAct多卡并行

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。