无需标注数据，让智能体自我优化工具链。
Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
- 用历史任务轨迹自动生成挑战性任务集并重试
- 单轮优化使SWE-Bench Pro通过率从59%提至78%
- 适合长期部署的AI代理持续进化
AI代理依赖技能、工具与工作流组成的工具链来解决复杂问题。持续改进该工具链对适应新任务至关重要。然而，现有优化方法通常需要真实标签验证集，而实际部署中此类标注数据难以获取。为此，我们提出回顾式工具链优化（RHO），一种仅使用历史轨迹的自监督方法。RHO从过往轨迹中选取多样化的高难度任务，平行重解这些任务。代理通过自验证与自一致性分析回放结果，生成候选工具链更新，并基于自身成对自偏好选择最优方案。我们在软件工程、技术工作和知识工作三个领域评估RHO。显著的是，单轮优化使SWE-Bench Pro通过率从59%提升至78%，且无需外部评分。分析表明，RHO能有效针对过往失败模式。优化后的工具链改变了代理行为模式，在长时程会话中维持更高准确率。
原文摘要 · Abstract (English)
AI agents rely on a harness of skills, tools, and workflows to solve complex problems. Continually improving this harness is essential for adapting to new tasks. However, existing optimization methods typically require ground-truth validation sets, yet such labeled data is difficult to acquire in practical deployment settings. To address this problem, we introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimizes the agent harness using only past trajectories. Specifically, RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel. The agent analyzes these rollouts using self-validation and self-consistency, then generates candidate harness updates and selects the most effective one by its own pairwise self-preference. We evaluate RHO across three diverse domains, spanning software engineering, technical work, and knowledge work. Notably, a single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading. Furthermore, our analysis demonstrates that RHO effectively targets prior failure modes. As a result, the optimized harness alters the agent's behavior patterns and sustains higher accuracy during long-horizon sessions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。