自动优化大模型智能体配置，无需编码即可提升性能。
Evolving Excellence: Automated Optimization of LLM-based Agents
- 用语义感知的遗传算法协同优化提示、工具描述等组件。
- 在多个任务上实现10%~37%的性能提升，最高达36.9%降本。
- 支持商业与本地模型，适合想快速调优智能体的开发者。
基于大语言模型（LLM）的智能体系统在自动化复杂工作流方面潜力巨大，但常因配置不佳而表现欠佳——提示词、工具描述和参数通常需数周手动调优。现有方法或过于复杂，或孤立优化各组件，忽略关键依赖关系。本文提出ARTEMIS，一个无需代码的进化优化平台，通过语义感知的遗传算子联合优化智能体配置。仅需基准脚本和自然语言目标，ARTEMIS即可自动发现可配置项，从执行日志中提取性能信号，并演化出更优配置，无需修改架构。我们在四个代表性智能体系统上评估：在AtCoder启发式竞赛中的ALE Agent，接受率提升13.6%；在SWE-Perf上的Mini-SWE Agent，性能显著提高10.1%；在Math Odyssey中的CrewAI Agent，评估所需令牌数减少36.9%；以及基于Qwen2.5-7B的小型开源模型的MathTales-Teacher Agent，在GSM8K上准确率提升22%，验证了其对商用与本地模型的通用优化能力。
原文摘要 · Abstract (English)
Agentic AI systems built on large language models (LLMs) offer significant potential for automating complex workflows, from software development to customer support. However, LLM agents often underperform due to suboptimal configurations; poorly tuned prompts, tool descriptions, and parameters that typically require weeks of manual refinement. Existing optimization methods either are too complex for general use or treat components in isolation, missing critical interdependencies. We present ARTEMIS, a no-code evolutionary optimization platform that jointly optimizes agent configurations through semantically-aware genetic operators. Given only a benchmark script and natural language goals, ARTEMIS automatically discovers configurable components, extracts performance signals from execution logs, and evolves configurations without requiring architectural modifications. We evaluate ARTEMIS on four representative agent systems: the \emph{ALE Agent} for competitive programming on AtCoder Heuristic Contest, achieving a \textbf{$13.6\%$ improvement} in acceptance rate; the \emph{Mini-SWE Agent} for code optimization on SWE-Perf, with a statistically significant \textbf{10.1\% performance gain}; and the \emph{CrewAI Agent} for cost and mathematical reasoning on Math Odyssey, achieving a statistically significant \textbf{$36.9\%$ reduction} in the number of tokens required for evaluation. We also evaluate the \emph{MathTales-Teacher Agent} powered by a smaller open-source model (Qwen2.5-7B) on GSM8K primary-level mathematics problems, achieving a \textbf{22\% accuracy improvement} and demonstrating that ARTEMIS can optimize agents based on both commercial and local models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。