构建可扩展的智能体环境平台,推动真实世界应用落地。
ARE: Scaling Up Agent Environments and Evaluations
- 用简单抽象构建多样复杂环境,支持真实与合成应用集成
- Gaia2基准测试显示强推理常牺牲效率,预算扩容渐趋平缓
- 平台支持持续扩展,适合研究新领域评估体系的团队
我们提出Meta Agents Research Environments(ARE),一个用于可扩展环境构建、合成或真实应用集成以及智能体编排执行的研究平台。ARE提供简洁抽象,支持创建具有独立规则、工具、内容和验证机制的多样化环境,有助于弥合模型开发与实际部署之间的差距。我们还基于ARE构建了Gaia2基准,用于衡量通用智能体能力。该基准不仅要求搜索与执行,还需处理模糊性与噪声、适应动态环境、与其他智能体协作,并在时间约束下运行。与以往基准不同,Gaia2采用异步运行,暴露出静态设置中无法发现的新失败模式。实验表明,在智能谱系上没有系统全面领先:更强推理常以效率为代价,预算扩容曲线趋于平缓,凸显需要新型架构与自适应计算策略。更重要的是,ARE的抽象设计使Gaia2能持续拓展至其他环境,赋能社区快速构建针对特定领域的新型基准。在人工智能的第二阶段,进展越来越依赖于定义有意义的任务与稳健评估以推动前沿能力发展。
原文摘要 · Abstract (English)
We introduce Meta Agents Research Environments (ARE), a research platform for scalable creation of environments, integration of synthetic or real applications, and execution of agentic orchestrations. ARE provides simple abstractions to build complex and diverse environments, each with their own rules, tools, content, and verifiers, helping to bridge the gap between model development and real-world deployment. We also propose Gaia2, a benchmark built in ARE and designed to measure general agent capabilities. Beyond search and execution, Gaia2 requires agents to handle ambiguities and noise, adapt to dynamic environments, collaborate with other agents, and operate under temporal constraints. Unlike prior benchmarks, Gaia2 runs asynchronously, surfacing new failure modes that are invisible in static settings. Our experiments show that no system dominates across the intelligence spectrum: stronger reasoning often comes at the cost of efficiency, and budget scaling curves plateau, highlighting the need for new architectures and adaptive compute strategies. Perhaps more importantly, ARE abstractions enable continuous extension of Gaia2 to other environments, empowering the community to rapidly create new benchmarks tailored to their domains. In AI's second half, progress increasingly depends on defining meaningful tasks and robust evaluations to drive frontier capabilities forward.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。