让智能体的运行环境自动学习修复失败，提升任务成功率。
Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
- 用在线强化学习训练专门的修复工程师，根据失败轨迹生成可执行补丁。
- 在多个任务中将基础模型成功率从44.3%提升至53.6%，微调后达64.2%。
- 适合需要持续优化运行环境的长期部署智能体系统使用。
基于大语言模型构建的智能体在部署过程中不断积累交互轨迹，但其行为通常保持固定。除了更新模型权重外，这些轨迹还可用于改进构建上下文、调度工具、验证动作并恢复执行的智能体运行环境。我们提出Harness-R1，据我们所知首个将失败条件下的全生命周期运行时编辑作为可学习能力的方法。通过在线强化学习对专用的运行环境工程师进行后训练，使其编辑决策以实际任务成功为优化目标，而非依赖固定规则。一个独立的90亿参数工程师将一批目标智能体失败案例转化为可验证的可执行补丁；同一批次的重新运行（冻结目标智能体）提供结果奖励，仅更新工程师。冷启动监督微调初始化该编辑策略，随后采用组相对策略优化进行在线训练。在WebShop、ALFWorld和DBBench三个基准上，Harness-R1将原始Qwen3.5-9B的成功率从44.3%提升至53.6%（+9.3个百分点）。在直接微调目标智能体后，特定于目标的工程师进一步将平均成功率从59.2%提升至64.2%（+5.0个百分点）；由于增益在目标微调前后均成立，表明运行环境工程师与目标智能体存在协同进化潜力。
原文摘要 · Abstract (English)
Agents built around large language models continually accumulate interaction trajectories during deployment, yet their behavior typically remains fixed. Beyond updating model weights, these trajectories can improve the agent harness that constructs context, mediates tools, validates actions, and recovers execution. We introduce Harness-R1, the first method, to our knowledge, that makes failure-conditioned, lifecycle-wide editing of an existing executable runtime a learned capability. It post-trains a dedicated harness engineer with online reinforcement learning so that its edits are optimized for the realized task success they produce, rather than proposed by a fixed editor. A separate 9B engineer converts batches of target-agent failures into validated executable patches; fresh same-batch reruns of the frozen target provide outcome rewards, so training updates only the engineer. Cold-start supervised fine-tuning initializes this editing policy, which is then trained online with group-relative policy optimization. Across WebShop, ALFWorld, and DBBench, Harness-R1 raises vanilla Qwen3.5-9B success from 44.3% to 53.6% (+9.3 percentage points). After direct target-agent fine-tuning, a target-specific engineer raises the average further from 59.2% to 64.2% (+5.0 points); because these gains hold both before and after fine-tuning the target, Harness-R1 points toward co-evolving the harness engineer and the target agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。