提出VeRO框架，系统优化代码智能体的迭代改进过程。
VeRO: A Harness for Agents to Optimize Agents
- 构建版本化、带预算控制的执行追踪框架，支持智能体代码迭代优化。
- 在多任务测试中验证不同优化器效果，发现特定修改可稳定提升性能。
- 适合研究智能体自我进化与代码生成优化的研究者使用。
编码智能体的一个重要新兴应用是智能体流程优化：通过编辑和评估目标智能体的代码实现迭代改进。尽管该任务具有重要意义，但社区尚缺乏对编码智能体在此任务上表现的系统性理解。流程优化不同于传统软件工程：智能体流程同时包含确定性代码与随机的大语言模型补全，需要结构化地记录中间执行轨迹和下游结果。为此，我们提出（1）VeRO（版本管理、奖励机制与观测记录），一个外层流程框架，提供目标流程的版本快照、预算控制的评估以及结构化执行轨迹；（2）VeRO-Bench，一套包含目标智能体与任务的基准测试集，配备参考评估流程。利用VeRO，我们开展了跨任务的优化器对比实验，并分析哪些修改能可靠提升目标智能体流程。我们已开源VeRO，以支持编码智能体作为核心能力的优化研究。代码见https://github.com/scaleapi/vero。
原文摘要 · Abstract (English)
An important emerging application of coding agents is agent harness optimization: the iterative improvement of a target agent by editing and evaluating its code. Despite its relevance, the community lacks a systematic understanding of coding agent performance on this task. Harness optimization differs from conventional software engineering: agent harnesses interleave deterministic code with stochastic LLM completions, requiring structured capture of both intermediate execution traces and downstream outcomes. To address these challenges, we introduce (1) VeRO (Versioning, Rewards, and Observations), an outer harness that provides versioned snapshots, budget-controlled evaluation, and structured execution traces of target harnesses, and (2) VeRO-Bench, a benchmark suite of target agents and tasks with reference evaluation procedures. Using VeRO, we conduct an empirical study comparing optimizers across tasks and analyzing which modifications reliably improve target agent harnesses. We release VeRO to support research on agent optimization as a core capability for coding agents. Code is available at https://github.com/scaleapi/vero.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。