针对智能体任务失败,自动识别短板并生成精准训练环境。
TRACE: Capability-Targeted Agentic Training

- 通过对比成功与失败轨迹,定位模型缺失能力。
- 在 $τ^2$-Bench 和 SWE-Bench 验证中分别提升15.3和15.0点。
- 仅用四分之一数据量即超越强基线,适合资源受限场景。
模型常因缺乏目标环境所需的核心能力而无法完成智能体任务。现有方法或直接微调目标环境,或生成非针对性合成数据,导致样本效率低、泛化能力差。本文提出 TRACE(将重复失败转化为能力导向训练环境),一个端到端的环境特定智能体自提升系统。TRACE 通过对比成功与失败轨迹,自动识别缺失能力,为每种能力构建奖励其是否被使用的合成训练环境,使用强化学习在每个环境中训练 LoRA 适配器,并最终融合多个能力适配器形成专家混合模型。TRACE 可跨环境应用,在 $τ^2$-Bench 上提升 15.3 点,在 SWE-Bench Verified 上提升 15.0 点 Pass@1。相比最强外部基线 GEPA 与 SWE-RL,分别领先 8.6 与 8.4 点。此外,其样本效率显著更高:仅需少于四分之一的回放次数,便超越 GRPO 与 GEPA,分别在 $τ^2$-Bench 上实现 +10.4 与 +8.6 点的最终准确率提升。
原文摘要 · Abstract (English)
Models often fail to complete agentic tasks because they lack core capabilities required by the target environment. However, mainstream approaches for addressing these failures typically either fine-tune directly on target environments or generate synthetic data that is not targeted to the model's actual capability deficits, resulting in low sample efficiency and limited generalization. We introduce TRACE (Turning Recurrent Agent failures into Capability-targeted training Environments), an end-to-end system for environment-specific agent self-improvement. TRACE contrasts successful and failed trajectories to automatically identify missing capabilities, synthesizes a targeted training environment for each capability that rewards whether the capability is exercised, trains a LoRA adapter via reinforcement learning on each synthetic environment, and then trains a mixture-of-experts model over the capability adapters. TRACE can be effectively applied across different environments, improving over the base agent by +15.3 points on $τ^2$-Bench, a customer-service agent benchmark, and by +15.0 points Pass@1 on SWE-Bench Verified, a software-engineering benchmark. TRACE outperforms the strongest external baselines, GEPA and SWE-RL, by +8.6 points and +8.4 points, respectively. In addition, TRACE is more sample-efficient than strong fine-tuning baselines: using fewer than one-fourth the number of rollouts, TRACE outperforms the best-performing baselines, GRPO and GEPA, and achieves higher final accuracy by +10.4 and +8.6 points on $τ^2$-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。