自动化设计智能体时，元智能体存在学习效率低、行为单一和成本不划算的问题。
Inefficiencies of Meta Agents for Agent Design
- 用进化策略替代累积历史设计，提升元智能体学习效果
- 设计的智能体行为多样性差，难以互补使用
- 仅在两个数据集上，自动设计才比人工设计更经济
近期研究尝试用元智能体自动设计智能体系统，通过迭代提出并优化新架构。本文分析了这类元智能体的三个关键问题：首先，发现简单地将所有历史智能体纳入上下文反而降低性能，而采用进化方法能显著提升效果；其次，尽管训练中生成多个智能体，但测试时通常只部署单一模型，其行为多样性低，限制了协同潜力；最后，评估自动化设计的经济可行性，结果表明仅在两个数据集上，当部署规模超过15,000个样本时，自动设计与部署的总成本低于人工设计；其他数据集即便扩大规模，性能提升也不足以抵消设计成本。
原文摘要 · Abstract (English)
Recent works began to automate the design of agentic systems using meta-agents that propose and iteratively refine new agent architectures. In this paper, we examine three key challenges in a common class of meta-agents. First, we investigate how a meta-agent learns across iterations and find that simply expanding the context with all previous agents, as proposed by previous works, performs worse than ignoring prior designs entirely. We show that the performance improves with an evolutionary approach. Second, although the meta-agent designs multiple agents during training, it typically commits to a single agent at test time. We find that the designed agents have low behavioral diversity, limiting the potential for their complementary use. Third, we assess when automated design is economically viable. We find that only in a few cases--specifically, two datasets--the overall cost of designing and deploying the agents is lower than that of human-designed agents when deployed on over 15,000 examples. In contrast, the performance gains for other datasets do not justify the design cost, regardless of scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。